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STUDENTS IN EXPERIMENTAL LARGE GROUP-SMALL GROUP 
CHEMISTRY CLASSES WERE COMPARED TO STUDENTS IN CONVENTIONALLY 
organized CHEMISTRY CLASSES FOR GAIN IN KNOWLEDGE OF 
CHEMISTRY AND OTHER LEARNING OUTCOMES. THE 1,241 STUDENTS 
involved IN THE INVESTIGATION WERE ENROLLED IN 14 URBAN, 
SUBURBAN, AND RURAL SCHOOLS. EXPERIMENTAL CLASSES RANGED IN 
SIZE FROM re TO 153 STUDENTS FOR LARGE GROUP INSTRUCTION AND 
FROM 7 TO 32 FOR SMALL GROUP ACTIVITIES. DATA WERE OBTAINED 
FROM STUDEN^ RECORDS, THE REGENTS EXAMINATION, AND TESTS 
constructed for the STUDY. STUDENTS WERE PRETESTED FOR 
KNOWLEDGE OF CHEMtSTRY, SCIENCE INTEREST, SCIENCE REASONING 
AND UNDERSTANDING, AND SCHOLASTIC ABILITY. THEY WERE TESTED 
FOR ACHIEVEMENT AFTER EACH OF SIX UNITS AND AT THE END OF THE 
YEAR FOR SCIENCE INTERESTS, AND SCIENCE REASONING AND 
UNDERSTANDING. DIFFERENCES IN THE EFFECTIVENESS OF THE TWO 

patterns OF instruction were determined by analysis OF 

COVARIANCE. REGRESSION ANALYSIS WAS USED TO RELATE SUCCESS OF 
DIFFERENT TYPES OF STUDENTS TO THE TWO PATTERNS OF 
INSTRUCTION. ACHIEVEMENT OF THE EXPERIMENTAL STUDENTS WAS 
SIGNIFICANTLY HIGHER THAN ACHIEVEMENT OF STUDENTS IN 
CONVENTIONAL GROUPS ON THE REGENTS EXAMINATION AND THE UNiT 
TESTS. (AG) 
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FOREWORD 



In addition to being a research undertaking in an important area of 
curriculum, the study of Large Group Instruction in Chemistry represents an 
innovation in the support and design of educational jfesearch. It is a successful , 
demonstration of a thesis, vigorously encouraged by Dr. Lome H. Woollatt, 
Associate Commissioner of Education for Research and Evaluation, State 
Education Department of New York, that cooperative educational research can 
produce more significant and more far-reaching results than research limited 
to single schools and single investigators. To be exact, the study shows that 

several schools with comparable programs provide a more diversified testing 

% 

ground for educational research than does a single school; that university 
professors trained In research methodology make a contribution to curriculum 
research which can not easily be matched by school p^Ysonnel alone; and that 
given cooperative support, both financial and moral, enough patience and good- 
will, and a good amount of just plain effort, a large Scale project can obtaih 
results which elude the less sensitive designs of smaller studies. 

Exactly where the idea for this cooperative curriculum research project 
arose is unknown to the editor of this report because he joined the project after 
some important early decisions had been made. (And until this day of reckoning, 
when the report is finally being written, he was more concerned with the present 
and the future of the research than with its past.) His apologies to those whom 
he might slight throng ignorance. When he joined project, the study of 
Large Group -Small Group Instruction in Chemistry Con^ared to Conventional 



Instruction had been agreed upon, Drs. Oerald T. Kowitz and Henry Hausdorff 
of the State Education Department had assembled a group of twelve school 
systems willing to support it, and Dr, John D. Scheller, of "Amherst Central, 
had agreed to serve as the coordinator for their contribution. The principal 
investigators were asked to submit a proposal for a research design which, 
after careful review, met the approval of the Department. .Thus, with the 
leadership of Dr. Scheller and much help from Drs. Kowitz and Hausdorff, the 
study got off to the happy beginnings which it was able to maintain to the very 
end. Perhaps the only unfortunate note that crept in during the three years of 
work was the realization, as datum followed datum, as punch-card box was 
stacked on top of punch-card box, and as the computer printed and printed and 
printed, that in our inexperiencb with a shidy of this magnitude we had under- 
estimated the wealth of material at hand. We had promised too little, too soon. 
This report makes only restricted uses of the data; t^ose* that were fully 
planned in advance to answer the questions posed in the initial prbposal. 

The principal investigators for the study were Drs. John J, Montean and 
John A. Schmitt of the University of Rochester and Drs. S. DaiHd Farr "and 
Stephen S. Winter of State University of New York at Buffalo, Dr, Winter also 
served as Project Coordinator and as editor of this report. Drs, Montean and 
Winter had major resppnsibilities for the chemistry-related problems of the 
research; Drs. Farr and Schmitt were chiefly concerned with evaluation, 
research design and analysis* In all this, they depended heavily for ideas and 
personal evaluations on the teachers in whose classrooms the study was 
carried out; 

Mr. George Martiny Akron Central Hi^ School 

Miss Esther B. Toinlinson Alden Central Ri^ School 

Mr. Elton W« Petersen^ Amherst Central Senior Hi^ School 
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Dr. Samuel W, Bloom 
Mr . Har<?M L, Kruger 
Mr* Lewis A. McCall 
Mr. Arthur H. Root 
Mr. Ben Varco 
Mr, Leonard S, Blake 
Mr. Lawrence Gideon 
Mr. Albert A, Deney 
Mr, Frank J. TuzSsqlino 



Benjamin Franklin High School 
Britton School 
Canandaigua Academy 
Clarence Central School 
Eden Central High School 
Kenmore East Hig^ School 
Orchard Park Central High School 
West Seneca Central Hi^ School ^ 
Williamsvllle Senior High School 



These teachers contributed items for the tests and provided tables of 



specification for them; they tried the instruments and criticized them; and 
always, they were willing to work sudden and difficult requests into their 



teaching schedules. In the final year of the study, when the pace became more 



o i>apid but Mso somewhat more predictable, they were Joined by: 



Mr. Leonard Weiss 
Mis/3 Beatrice Elye 
Mrs. Camille Gilmour 



Cleveland Hill High ^hool 
Cleveland Hill High ^hool 
Hamburg Central High School 



To all of them the principal investigators express their thanks, and also their 
admiration for the good nature and good will maintained consistently throughout 
leisure and chaos alike. 



The study owes much to four research assistants, all graduate students 
at State University of New York at Buffalo. Messrs. James Blaydon, Jack 
DuMey, Richard Egelston, and William Martin likewise reftised to get rattled or 
upqet or even admit to being tired when testing from 7 A.M. until 4 P.M. at 
schools miles apart, when sitting in front of computers that were unable to 
stand the heat and humidity of a midsummer weekend midnight and failed to 
compute as scheduled, or when the papers piled hi^ on their desks threatened 
to crowd them out of their rooms. As representatives of the study office who 
most frequently visited the schools, they helped maintain the happy 
communications. 

Much help was given also by a number of other persons: Mrs. Gail 
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Hofmann and Mrs. Sandra Yarnes, our secretaries; the guidance c|ounselors, 
principals, and their secretaries at the schools; Mr. Howard English, of the 
computer center, Mr, Robert C. Kochersberger, To all of them our thanks. 
For financial support of the study we are indebted to the State Education 



Department and the Boards of Education of the participating school districts. 



The Department, and the chief school officers of the participating school 

\ 

districts; 



Mr, Edward E. Allen 
Dr, Wilson R. Conrad 
Dr. John D. Scheller 
Mr. Fred B, Painter 
Mr. Robert Helmer 
Dr. Arthur Shedd 
Mr. Walter J. Heffley 
Mr, Edwin C. Peck** 

Dr, Richard Burau 
Mr. Harry H. Hatten 
Dr. Carl W. Baisch** 

Mr. C. Sherwood Miller 
Mr. Elmer D. Handel 
Dr, Robert Springer* 

Mr. Herman Goldberg 
Mr, Alfred W. Goodreds 
Dr. William E. Keller 

* deceased ** retired 



Supervising Principal 
Supervising Principal 
Supervising Principal 
Supt, of Schools 
Supt. of Schools 
Supt, of Schools 
District Principal 
District Principal 
District Principal 
Superintendent 
Supt, of Schools 
Supt. of Schools 
District Principal 
Supt. of Schools 
Supt. of Schools 
Supervising Principal 
Supervising Principal 



Akron Central H. S. 

Alden Central School 
Amherst Central Sr. H. S, 
Brighton H. S, 

Canandaigua Academy 
Clarence Central H, S, 
Cleveland Hill H. S. 

Eden Central School 
Eden Central School 
Hamburg Central ^hool 
Kenmore Public Schcols 
Kenmore Biblic Schools 
Orchard Park Central H. S, 
Rochester Public Schools 
Rochester Public Schools 
West Seneca Central School 
Williamsville H. S. 



provided much more than funds, however, through their constant interest and 
encouragement. 



And finally, to the students who so. willingly participated in this experiment, 
who subjected themselves cheerMly, if not voluntarily, to several three-hour 



blocks of solid testing, our most sincere appreciation. We hope they feel that 
their cooperation will lead to better education for their younger brothers and 
sisters, and for their children. 

This report has been written by all four principal investigators. As editor, 
however, I cannot disclaim major responsibility for inadvertent shortcomings, 
Buffalo, New York Stephen S. Winter 
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Chapter 1 
INTRODUCTION 

Large group instruction is a relatively recent phenomenon in American 
secondary education. Its origins are not entirely clear from the literature; 
however, there is little doubt that the work of the Commission on the 
Experimental Study of the Utilization of Staff in ihe Secondary School, whose 
director was J. Lloyd Trump, gave impetus to the spread of this pattern of 
instruction. The Commission’s report, GUIDE TO BETTER SCHOOLS (Trump 
and Baynham, 1961), gave the detailed rationale for the adoption of methods of 
instruction \\hioh brought together groups of 300 pupils for purposes that could 
be achieved as well with groups of that size and complemented this with smaller 
groups that ranged in size down to supervised instruction of one pupil at a time 
for purposes in which the sole learner had to be segregated. Among the, 
objectives of the report seems to have been an attempt to break the habitual 
25-30 pupils per class which has stifled flexibility in the deployment of 
instructional resources in American secondary schools. 

While the Trump report drew upon selected explorations with large and 
small instructional groups, at the time of its publication few empirical studies 
of the results of such changed practices in the secondary schools had been 
completed. What little evidence existed w^s chiefly in the form of testimonials 
or of uncontrolled measurements of achievement. Such evidence is suspect, 
and far short of desirable. However, these relatively unsophisticated studies 
invariably were favorable to the large group - small group organization on 
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one or another criterion. Thus there was good indication that large group 
instruction did not hurt youngsters* achievement of content; that it was pleasant 
to them, or at least not traumatic; and that it was viewed favorably- also by the 
teachers who tried it* For instance, Lisonbee (1962) reported the following 
personal observations concerning a large group class in chemistry: discipline 
was no problem and rapport between instructor and students was excellent 
throughout; student participation increased as time went on and it became 
possible to have class discussions; personal help could be given to students in 
small groups; 95% of the students liked the course, although ‘*90% voted 
equally for, the large class and a regular one.” Finally, in the important area 
of achievement, “it was Indicated that the experimental group held their own 
and showed a probability of doing slightly better than expected,*’ A dissertation 
by Breedlove (1963) generally confirmed these results adding that large group 
students gain greater undeipstanding of the methods of science, 

Lisonbee and Breedlove’s studies deal specifically with chemistry, the 
subject of the present investigation. They are fairly typical of other reports, 
however. Thus Johnson and Lobb (1961) report on the effect of class size on the 
** achievement, attitudes, and behavior of the learners** in Jefferson County, 
Colorado: 

I 

Asj a result of this study it was determined that the 
si^e of class did not in Itself make any significant 
. dij^erence. Specifically, the experiment produced 
tl^se findings; first, there was no significant 
difference in the/ achievement -of pupils in classes 
of 20, 30, 60, and 70; second, small groups of high 
capacity leaj^ners were not academically or 
economically/ueasible; and further, students had not 
^en harmed by participating in large group work. 

Other research of similar nature, with the same kinds of results, has been 

reported in the annual issue of the Bulletin _ of the National Association of 
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Secondary School Principals^ devoted to' team teaching and large group 
instruction. 

At the college leveli where the lecture has been the traditional method 

of instruction, more extensive investigations of large group instruction have. 

been conducted. While some of these were somewhat more adequate in desi^ 

than mod^igh school studies, an exhaustive review of the literature by Fleming 

(1958) eame up with the so common, yet so discouraging, result in educational 

research; no significant difference. To be exact; 

Once again, therefore, it has to be said . . , that 
class si25e itself does not appear to be a significant 
factor in achievement, and further support is lent to 
the (idea) ... that the secret difference between the 
success of one teacher must be looked for in some- 
thing other than the mere size of the group under 
tuition. 

Since 1961, additional reports have come in of the effectiveness of large 
group instruction. In general, they reinforce the conclusions which were evident 
when this study was planned in the fall of 1961. 



The Advantages of Large Group Instruction 

\ 

While large group instruction had not proved itself advantageous — nor 
disadvantageous--in terms of achievement, there sure anumberof other factors 
to be considered. These are given by Trump and Baynham (1961, p. 30-31): 

In tomorrow’s schools, more students will be 
exposed to skilled teaching in all subjects because 
the most capable and experienced teachers in 
specific fields will teach large classes. Every 
teacher, inevitably, is more experienced in one 
subject or one phase of a subject than in another. 

So the students can be better motivated by contact 
with the very best teacher available for that phase 
of the subject. The large class will' avoid the 
duplication of effort required when teachers must 
teach the same subject matter to a number of 
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classes, as in today^s schools. 

Tomorrow’s schools will find a number of 
other advantages in large class instruction. It makes 
technological equipment more economicaliy fe^ible. 
Many schools which can’t afford these aids for five 
or six custornary-size classes will be able to equip 
a single large classroom. 

Ecopomy^ — in this case economy of time— will 
allow tomorrow’s schools to schedule more 
presentations by outside specialists, by teachers 
from nearby universities who are ejqjerts in certain 
fields, and by communi^ resource persons. For 
example, in one of the experimental schools, a local 
• physician came to the school and talked at one time 
to all of the biology students aboiit the circulation of 
blood. His experiences as' a surgeon gave him more 
background about the heart and blood system than, 
any teacher in the schoo^ possessed. The crowded’ 
days of most of these p&>ple make it difficult for 
them to leave their own work for more than an hour 
or two at a time. 

For students, large classes will offer another 
particularly important advantage; They will serve 
as transitional experiences for college classes and 
for many other occasions of adult life. Students can 
learn to, take notes, hold back questions until an 
appropriate time, and develop more responsibility 
for planning their own learning. 



Rationale for /This Study 

The advantages cited by Trump and Baynham are indicated by logic and 
supported by the testimonials which had been published at the time this study 
was planned. The principal investigators in this research felt, however, that 
through an adequate plan for research, some of the indications could be 
corroborated by firm evidence and some questions which had not been raised 
before could be explored as well. To be exact, it was felt that a massive and 
carefully planned research effort could go beyond the gross measures of 

t'' . 

achievement which had typically resulted in ‘^no significant difference” to 
explore relationships betwe*Sn the competence of the learner, the method of 
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instruction, and the cognitive caitcome achieved (Bloom, 19S6). Secondly, It 
was thougiit that questions relating to affective learnlngs—attitudes, 
motivations^ — which had been explored chiefly In the college setting and for 
which differences were recorded (Gerberich and Warner, 1936; Ruja, 1954; 
Bloom, 1953) could also be explored, and through more sophisticated approaches 
than the direct questionnaire. Finally, the claims made for equal or lesser 
cost under the large group - small group pattern while adding to the teacher’s 
preparation time were felt in need of empirical verification. Thus the study was 
planned to supplement wh"at had been reported before and to seek evidence in 
aspects of the large group organization which heretofore had been unresesrched. 

A last dimension which required empirical study was the character of 
large group -small group instruction. The term “large group'- small group 
Instruction” is too Imprecise to serve as an adequate description of the learning 
situation. More accurate descriptions of the teaching tactics used both under the 

o 

t - ' ~ ' ■ 

conventional and under the large group - small group pattern were felt necessary 
to expMn any differences than might be discovered. Consequently, the study 
included ait e^densive exploration of the methods used in each of the classrooms 
so that learning interactions as well as group size could be examined as 
correlates of achievement or motivational results. 

Time Schedule 

The study was begun late in the fall of 1961 and planned for completion in 
1964, Fortunately, no major deviations were required until the very end when a 

m ^ * fr 

number of factors delayed data analysis and evaluation, and the production of 
the final report. 

■ ' . ■ ^ 

During 1961-62, it Was planned to identify, the instruments needed for the 
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study and to begin their -construction, testing, and evaluation. The year was 
also used to develop the communications processes neededfor this study which 

involved teachers at 14 schools and two universities* 1962-63. was set aside to 

' ^ / 

complete instrument production and to test research methodology and 
procedures with a fraction of the schools and a small block of chemistry 
content and teaching time. Finally, the complete study was carried out during 
the entire school year 1963-64, beginning with a teacher administered pretest 
on the first day of instruction and ending with the Hegents Examination late 



in June 



Chapter 2 

DESIGN AND INSTRUMENTS 



The , empirical investigation of the results of large group - small group 
instruction in Regents chemistry as compared to conventional instruction set 
itself the followng goals; , 

1. To determine which organizational pattern produced greater achieve- 
ment after allowance had been tnade for differences among the pupils through 

I 

an analysis of covariance treatment; 

a. In content as measured by the Regents Examination. 

b. In content as measured on tests of six common units. 

c. In content as differentiated according to a classification scheme of 
competencies in the cognitive domain following that of Bloom (1956). 

d. In science interest. 

e. hi science reasoning and understanding. 

2. ' To identify the kind of learners for whom the above achievement 
statements might be true. This part of the study attempted to answer ctuestions 
such as: Do girls achieve more hi^y on memorative content materials when 
tau^t in large groups? Do students with greater critical thinking ability change 
toward greater science interest when taught by conventional organization? Do 
students with a poorer academic record achieve more highly in one or the other 
organizational pattern? 

3. To ascertain whether organizational pattern also identifies 
instructional procedures, or viiether instructional tactics with each pattern 



vary so much that teacher characteristics are more important than 
organizational pattern. ' 

4. To assess whether there are cost differentials that mi^t preclude 
adoption of the large group pattern— even if itproved educationally desirable— 
except in the most hi^y supported school districts. 

The design of the study, therefore, required a large number of 
meaSur^ent instruments some of which were available among the published 
tests, some of which, however, had to be produced for the study. 

Criterion Instruments 

^ o -- 

The following tests were used as criterion instruments; 

1. The Regents Chemistry Examination for 1964, to measure achievement 
in accordance with the Regents Chemistry Syllabus. 

(• 

2. Ihe Kuder Preference Record, Vocational Form CM, which has a 
science intea^est scale, to measure changes in science interest. 

3. The Test of Science Reasoning and Understanding produced by Dressel 
for the Cooperative Study of Evaluation in General Education of the American 
Council on Education to measure changes in science reasoning and 
understan<hng. 

These three instruments were used as available. In addition, achievement 
on six units of the Regents chemistry course was determined by six unit tests 
devised specifically for this study. Moreover, the items for these tests were 
constructed in such a manner that the^ represented the various categories of 

n ^ 

competencies in the cognitive domain so that the unit tests served also as 
measures of the different intellectual competencies. 



Control Instruments 






In order to characterize students and tosegi^gatethein^nto educationally 
meaningful groups, the study employed a large number of xnstrm^nts covering 
attributes presumed independently influential in determining success on the 
criterion variables. Among the dimensions explored, most 6f?ivhich^^sequentiy 
were shown to result in relatively independent scores, were intellectual 
competence and achievement; science interests; prior knowledge of chemistry 
and chemistry aptitude; study habits; critical thinking; science reasoning; and 
some socio-economic factors.. Most instruments needed for these purposes 
were available among published tests; 

1. The Holzinger- Crowder Unifactor Tests, v^ich have four parts (verbal, 
spatial, numerical, and reasoning), characterized the competencies usually 
associated with IQ. 



2. Students’ grades in* academic courses from grade 9 onward, obtained 
directly from records on file in the schools, characterized their general, 
scholastic achievement, 

3. Students’ science interests were char acterizedby the Kuder Preference 

Record administered as a pretest (as well as a posttest. See above) . ” 

4. Chemistry aptitude was characterized by the lov/a Placement 
Examinations, Chemistry Aptitude, Form M. Since this test was published in 

1942, its part 4, dealing with chemical facts, was replaced by a Chemistry 

■* , 

Achievement Pretest constructed especially for this study, 

*, ry 

5. Study habits of the students were characterized by the Brown-Holtzman 
Survey of Study Habits and Attitudes. 

6. Critical thinking was characterized by the Test of Critical Thinking, 
Form G, developed by the Cooperative Study of Evaluation in General Education 
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of the American Council on Education, 

3 • 

7 . Science Reasoning Ability was characterized, by the Test of Science 
Reasoning and Understanding (See -above) given as a pretest, 

8. Socio-economic factors were characterized by information supplied 
by the subjects of this study regarding their parents’ education and occupational > 
status.. 

It was found that not all of these instruments gave information regarding 
student competences vd?ich was sufficiently independent of the other instruments 
to serve as a useful characterization. The Iowa Placement Examination, The 
Survey of Study Habits and Attitudes, the Test of Critical Thinking and the 
socio-economic data were not subsequently used in the analysis. An additional 
characterization, of course, was sex of the student. 
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Other Variables 

The achievement of the third goal of this study, the description of 
teaching tactics used by each of the teachers throuj^out the year, required 
the construction of still another, instrument. The literature contains plentiful 
suggestion concerning the construction of check lists of teacher performance. 
These, however, have supervisory and evaluative purposes which were contrary 
to the objectives of the description instrument needed for this study. On the 
other hand, Flanders has devised a method for describing the classroom 
behaviors of teachers (Flanders, 1960). His method, however, requires an 
observer. This eliminated the Flanders instrument. A measure was developed 
for this study on which each teacher recorded his behavior in categories 
important to the description of te^hing tactics and which he could fill in daily 
with relatively little difficulty. 
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Finely, an analysis of teacher time use and space use served as the 
basis for the determination of teaching costs of the two programs. Professor 
Milton Pullen of the University of Rochester served as consultant to this part 
of the study. The cost analysis went, in part^ beyond the participating schools 
in order to extend the sample. 



The Unit Tests ■» 

At one of ihe first meetings of groups involved in this study, principal 
investigators and teachers, it was ascertained that each teacher teaches as a 
discTe^ unit the material in the Regents Syllabus related to each* of the 
following topics: 

1. The Periodic Table and the Properties of Atoms. 

2. Water, Solutions, and Ionization. 

3. Principles of Reaction.* 

4. The Halogen Elements, 

5. Elements of the Sulfur and Nitrogen Families. 

6. Organic Chemistry, 

♦Subsequently it was discovered that Principles of Reaction were not 
taught as a unit but merely reviewed as such. See Chapter 3. 

These six were therefore chosen as the units which could give. partial 

: ' J. 

information on student achievement and for which tests could be developed 
with questions that contained items written specifically in accordance with all 
categories of Bloom’s taxonomy (1956). Teachers were invited to submit old 
tests on these units to the principal investigators to serve as the source of 
items for the unit tests. With tiiese tests and the Regents- Syllabus as guides, 
tables of specifications were prepared for each of the unit tests. TheSe were 





' \ 

\ 

submitted to the teachers for an indication of the relative emphasis on each of 
the topics within the unit tests. Responses were averaged for the final 
distribution of items in accordance with the table of specifications. The tables 
of specification for the xmit tests, with an indication of the relative emphasis 
within each test, are given in Table I. 

, The principal investigators then wrote about 100 items of multiple choice 
type for each unit. The items were adapted from the items submitted by the 
teachers and roughly approximated in numbers the distributions with|n the 
tables of specification. Particular care was taken to include items which Icould 

'I 

• I 

be rated according to each of the intellectual competencies of Bloom. 

Several compromises had to be made in adapting Bloom's taxonomy to 
the study. In the first place, the taxonomy is quite detailed. It was recognized 
that only the major subdivisions of the taxonomy could be represented by a 
sufficient number of items to provide an adequate sample of that competence 
among the tests. Secondly, it was found that items requiring evaluation, 
analysis, or synthesis are extremely difficult to construct within the multiple 
choice style required for the reliable handling of the data in this study . Thirdly, 



Bloom's definitions of categories are relatively cumbersome and at times 
unclear. A simpler scheme for classifications was required. Accordingly, the 
following four categories were established for the purpose of this study: 



Type I. 



Recall: Any item which had been taught in substantially identical 
form, and which required merely the recall of that bit of 
information. Rephrasing, inversion of sentences, and similar changes 
of form were considered not to change an item from the recall 
category. Examples of recall items, from several of the tests, are: 



Which of the following < water solutions does not conduct electric 
current? • 



a) CuSO 

b) : 

c) HGl 



d) C^HgOH 

e) KNOg 
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TA^LE I 

TABLES OP SPECIFICATION FOR THE UNIT TESTS 



Periodic Table 



1> History 

2_^Principal sub- atomic particles, including atomic structure, 
atomic number, atomic weight, isotopes 

3, El'ectron levels (orbitals and spectra) ' 

4, Bonding ionic and covalent bonds, characteristics of 
compounds, electron-dot diagrams of simple compounds 

5, Valence, metals, non-metals, activity and electronic 
structure, activity and size 

6, Periodic Table, families, periodic variation, other 
relationships 

7, Symbols, formula writing, molecular weight from formula 



Water, Elutions, and Ionization 



1. Solutions— definitions, rate of solution, solubility, satu- 
ration, solvents 

2, Water crystallization, water of crystallization, dehy- 
dration, purification and distillation 

3* Ifieory of ionization, water solutions of electrolytes, molten 
, electrolytes, non- electrolytes, electrolysis 

4. Concentration (molar) and other 

5. Boiling point, freezing point changes, vapor pressure 
b. Acids, bases, neutralizing action, pH, hydrolysis 



Principles of Reaction 



1. - Factors causing completion of reaction (precipitatei eras, 

oxidation, etc.) » & 

2. Activity of metals, displacement series j displacement of 

some non-metals - 

3. Redox, agents, oxidation number 

4. Energy changes, stability, exo-endothermic 

5. Types of reactions 

6. Equilibrium and LeChatelier’s Principle, Common Ion 
Effect (no numerical problems) 

7. Velocity of Chemical Change and factors affecting it, 

catalysts . ‘ 
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17% 

10 % 



15% 

20% 



16% 

16% 



15% 

10 % 



25% 

18% 

7% 

25% 



20 % 



16% 

20% 

9% 
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6 % 
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TABLES OF. SPECIFICATION FOB THE UNIT TESTS 

(Continued) 

The Halogen Elements . . 

” 1, Family in periodic table) changes of properties in the family 12% 

2. Occurrence and production of elements > 12% 

3, Properties ^d reactions of the elements ' 23% 

4. HX compounds, production, uses; MX salts 11% 

5. Oxy-acids, names, oxidatiQn numbers; etc, 12% 

6, Redox in the halogen f amily and review of redox 13% 

7 , Review of wei^t, volume, composition problems 10% 



Sulfur and Nitrogeh 'Families 

1. Sulfur, forms, Fbasch, reactions 7% 

2* SO^ and H^SOg, preparation and reactions 11% 

3. preparations, reactions, acidity, test for 12% 

4. H2SO^ -SOg,. reactions, identification. Contact process 11% 

5. N^f sources, uses in combined form 7% 

6. Nt fixation, including Haber, Ostwald 8% 

7. Chemical equilibrium, review 8% 

8. HNO^ — properties, preparation, reactions 13% 

9. NHg — properties, preparation, reactions, weak base 12% 

10, Oxides of N 7% 

11. Other elements in family (P, etc.) 5% 



Organic Chemistry 

1. Definitions, tetravalent C* difficulty of reactions, etc. 7% 

2. Homologons series, series formulas, names 16% 

3. Structural formulas and isomerism 14% 

4. Unsaturation, saturation, and simple reactions — double, 

triple bonds X4% 

5. Benzene, simple reaction, simple derivatives 6% 

6. Functional groups (Orit, RCOOH, RCOOR) 16% 

7. Ethers, aldehydes g% 

8. Organic fuels—petroleum, gasoline, octane, etc. 10% 

9. Polymerization, and other reactions 7% 

10, Other (biologic applications, plastics, etc.) 4% 



Which of the following is a ^od reducing agent? 

a) S b) Brg c) d) C e) 

Type II, Comprehension; Any item whichrequiredtheapplicationof a principle 
under circumstances not identical with the context in ’^^hichthe 
principle had been taught, but in such form that the necessary 
principle is implied in the question. Examples of comprehension 
items are: 

The ion in which chlorine has an oxidation number of -1 is 

a) cr b) CIO' c) ClOf d) CIO 3 " e) CIO^' 

The atomic number of a certain element is 53. An atom of this, 
element must contain 

a) 53 neutrons 

b) 26 neutrons and 27 protons 

c) X neutron, 26 protons and 26 electrons 

d) 53 protons 

e) 53 particles of all kinds 

Type III, Application; Items as above except that the principle heeded is not 
implied in the question, and the student must choose the needed ^ 
principle as well as apply it for the solution. Most numerical problems . 
were considered in this category. Examples are: • 

o 

The reaction between lBa{OH )2 - HNOg 

a) approaches completion because a precipitate is formed. 

b) approaches completion because a gas is formed, 

c) approaches completion because an un-ionized substance is formed, 
dl approaches completion because two of the above both occur, 
e) does not ^roach completion; none of the above applies. 

AlgOg and CCl^ are the correct formulas of the oxide of. aluminum 

and the chloride of carbon. The formula of the compound aluminum 
carbide Is therefore 

a) Al^Cg b) AlgC^ c) Al^C d) Al^Cg e) Al 2 Cg 

Type IV. Higher Competencies; This category included items which required 
analysis of a complex situation and the subsequent drawing of 
analytical, synthetic, or evaluative inferences. Examples of this kind 
of item are; 

The following equations describe the reaction of Fe with HCl and 
2 Fe + 2HC1 — ^ ^'^©Clg + Hg 

2Fe + 3Clg^ — ^aFeClg 



This proves that 

a) Cl^ is a good reducing agent 

h) HCl is a good reducing agent 

c) HCl is a good oxidizing agent 

d) Clg is a better oxidizing agent than HCl 

e) CI2 is a better reducing agent than HCl 

As a 6N solution of Sulfuric acid is diluted to O.IN, the pH 

a) increases d) may increase or decrease 

b) decreases e) cannot predict the answer 

c) remains the saipe 

It was recognized in the construction of these items, as Bloom already 

» 

pointed oiit, that all items require an initial recall of information so that, in 

this sense, they depend initially upon rote memory. The distinction is between 

memory alone and memory plus other intellectual behaviors. Secondly, the 

distinction between categories is heavily • dependent on judgments on "ftbich 

consensus is not perfect. Thirdly, any item, whatever the ingenuity in its 
» & 
construction, can be turned into a recall itpm if the teacher chooses to. teach 

its content in that manher. Nevertheless, the length of the sequence of 

Information which has to be memorized differs for the various items and, even 

in that case, the categories distinguish at least among different complexities 

in the material memorized. 

Each of the items for each t^st was identified according to the aix)ve type 
by concensus between Drs. Montean and Winter after tabulation of ratings given 
the items by the collaborating chemistry teachers. The items were then split 
into two tests, one typically somewhat long but considered “easy** by the 

students ^o took it and containing chiefly Type 1 items— sometimes 

1* 

supplemented by a few Type H items— and the other containing all other items. 
As each participating teacher completed the unit for which the unit test had 
been constructed, he was given enou^ copies of each of the two tests for half 
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of his classes. He was not told which test was **easy’* and which *‘hand** but 
asked to distribute the tests among his pupils without any specific ordering. 
Teachers were asked to use the results from these tests in their grading to 
insure usual motivation^ and were promised* and subsequently sent, normative 

and scaling data for each of the two tests* ✓ ' 

/ 

The tests were returned to Buffalo Tor fading and item analysis. Item 
difficulties, item discrimination, and point-biserial correlation values were 
computed (GuHil^en, 1950). All items were then reviewed in a meeting of all 
teachers with the principal investigators to explore difficulties that might have 
arisen during administrationj problems with comprehension, applicability, 
style of response, etc. Only items which were agreed upon by all teachers, 
which discriminated ^significantly, and were answered correctly by 
approximately 40-^% of the students were retained* Prom this acceptable 
group of items; a final 50-item test was constructed for the study, with items 
distributed according to the table of specifications and containing approximately 
40% of Type I items and 20% of items of each of the other types. It was difficult 
to adhere to ttoth sets of restrictions, and distribution of item type was 
compromised in the final construction of tests (Table H). Two forms of each 
test were produced with identical items arranged in a different order. 

The Chemistry Pretest 

The Chemistry Pretest was produced in essentially the same fashion as 
the unit tests. Since It was designed to measure prior chemistry knowledge, 
Items were chiefly of Type I. Their distribution followed the topics of the 
Regents Syllabus in chemistry. Items were written by the principal 
investigators, sent to the schools for administration during the first week of 
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TABLE II 

DISTRIBUTION OF ITEM-TYPES AMONG UNIT TESTS 
TEST NAME NUMBER OF ITEMS OF 'TYPE 





I 


II 


in 


IV 


I^erlodic T^Sle 


15 


12 


10 


13 


Water, Solutions, and Ionization 


17 


15 


12' 


6 


Principles of Reaction 


12 


L, 

14 


18 


6 


Halogens 


15 


18 


8 


9 


Sulfur and Nitrogen 


22 


10 / 


8 


10 


Organic Chemistry 


14 


30 


5 


1 




95 


99 


61 


45 



the school year 1962-63, discussed in meeting, and analyzed. From the result 
of the trial administration, a 40-item test of items with the required response 
characteristics ahd following the distribution of the Regents Syllabus was 

constructed and called the Chemistry Achievement Pretest, This test also was 

* 

produced in two forms differing only in the sequence of the items. 

The Instructional Process Report Form 

The major purpose of the Instructional Process Report Forrn was to 
establish whether group size also implied a difference in the style of teaching. 
Consequently, the emphasis in the instrument was on the recording of the 
kinds of activities vhich the students experienced in each classroom, with an 
indication \^hether the objectives of the instructional activities differed in the 
two organizational patterns. 

The instrument consisted of a series of simple response categories giving 
time, place, size of. group, subject matter covered, kind' of session, length of 
session, kind of homework, etc. Then, there followed an open-response section 

in which teachers recorded, also by code, the specific kinds of interactions 

■ . t I ■ ■ 

that took place, whether teacher dominated, teacher-pupil exchange, pupil 

■ I. 

dominated. Also recorded was the planned purpose of the interactions and Iheir 
duration. Tliere were 23 categories under classroom interactions and 10 under 
objectives or purposes In addition, the form contained space for remarks or 
aspects not provided for among the encoded categories. 

This instrument, which proved quite functional when appropriately used, 
was developed through a series of trials of short duration followed by analysis 

** A 

of responses and discussion of results with the teachers who had to use it. 
For each trial, the principal investigators prepared a direction booklet for 
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the completion of the forms, with the stipulation that the forms would he used 
for one section of Regents chemistry in all its scheduled activities— large 
group, small group, and laboratory; or conventional class meeting and 
laboratory — ^and in all non-scheduled contacts with students which concerned 
the course. Among the latter were pupil-solicited or teacher-suggested help 
to individuals or small groups; make-up work; help with projects; and other 
similar non-scheduled cont^ts. Typically, each teacher used about three 
report forms daily, one for classroom and two for other teaching-related 
activities or contacts. The instrument ■ went throu^ three two-week trials 
followed by careful revisions before categories were considered adequately 
exclusive and the response form adequately simple to allow rapid and' 
unambiguous description of the teaching style. It was then used in the pilot 
study of approximately six weeks duration id which it proved satisfactory for 
extended use. 

Pupil Achievement and Socio-economic Data ' 

The gathering of data of pupil’s past achievement, surprisingly, proved 
more difficult than anticipated. There is great diversity in the way schools 
maintain their records. Consequently, it was necessary to provide the guidance 
office staff in each school with considerable detail regarding the needed 
information before it could be extracted from the filee without undue di^icuity. 
Through the experience gained in obtaining achievement and socio-economic 
data for the pilot study, it was possible to design a form that met the needs 
although strange course titles^ or occasional inversions in the order in which 
the courses* were offered still presented some difficulties. The form that was 
finally used allowed each student to check the courses he had taken in each 
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acadeir^c area in each grade from grade 9 on. Grades were later supplied 
for the checjced courses by the guidance officials or their secretaries. Also 
on the form, the students recorded parents* occupation, parents* education, 
and personal vocational objectives. 

Coat 

The assignment of specific costs', to Regents chemistry in the two 
organizational patterns proved to be the most difficult task confrCnting the 
study. School budgets are not kept in a manner that allows the easy isolation of 
costs attributable to one course. After thorough discussion of these matters 
with school business officials and principals, Professor Pullen, the consultant 
for this part of the study, recommended that the cqst study limit itself to 
determination of teacher time costs, space costs, and recorded supervisory 
and developmental costs. While this eliminated a number of cost categories 
from consideration because data were not attainable, it -was felt that this 
simplification of the study was justified because salaries make "up the major 
fraction of the school budget, space costs also rank high, so that only lesser 
cost factors were neglected. Moreover, no lo^cal basis was found for the 
assumption that laboratory or supply costs should differ significantly under 
the two plans. The ignoring of such factors meant, at worst, the ignoring of 

s 

small differenced among small Mdget factors. 

i 

Periods assigned to chemistry instruction rather than teacher pay was 
used as the basis for ooipparihg teaching costs because the salary scales in 
the participating districts were not strictly comparable and, furthermore, the 
exact cost of Instruction varies with factors that are not related to large ^oup« 
small , group or to conventional instruction. Henoe, it was considered more 
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important to know that one teacher instructed a total of 125 pupils per week 
than that a salary item of $7,000 performed such instruction. 

Ultimately, a very simple form was devised to attribute teacher time to 
Regents chemistry and to make corrections for the fact that the number of 
exposures to chemistry could be either six or seven periods per week for both 
kinds of schools. In Edition to teacher-pupil ratios, the form recorded 
spaces utilised and recent direct expenditures for developmental and 
supervisory costs. In order to extend the sample, *a number of achoolU not 
othervdse in the study were added to obtain more broadly representative data 
for the cost analysis. 



Chapter 3 
THE STUDY 



While a total of three years was required to complete the study of large 
group - small g;roup instruction in Regents chemistry compared to conventional 
instruction, the data for the. final comparisons were obtained during the yesuc* 
1963-64 only. All earlier work was preparatory and was concerned with 
selecting and developing the measurement instruments needed for the study, 
’ptfe'-^ting the efficacy of the instruments, with establishing the communications 
processes needed to collect the large amount of data easily and on time, and 
with testing the data reduction capabilities of the computers with the available 
. computing programs. Since . these preparatory activities had little direct 

bearing on the conclusions of the study, they will be treated very briefly. 

■ ' ■ ■■ 1 

1 

\ 

1961-62 

. The first year was devoted only to instrument selection and instrument 

construction. It was found, as has been recognized elsewhere (Watson and 
Cooley, 1960), that there is a severe paucity of good instruments for measuring 
interests, motivations, and critical abilities in science. Moreover, there are 
no tests witti validated items specifically categorized according to cognitive 
difficulty. Thus, the construction of unit tests widi items following Bloom’s 
teoconomy was immediately planned* and a later decision to construct a special 
Chemistry Achievement Pretest was based on the same reason. 

During the first year, work was begun on all six unit tests, and the tests 
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on Halogens, Sulftir-Nitrogen, and Organic Chemistry went thnouj^ pretesting, 
analysis, and evaluation. The other tests could not be pretested because the 
appropriate units had been tau^t before the trial tests could be constructed 
and had to be carried over into the second year of preparation. The Instructional 
Process Report Form went through two brief trials and revisions* The other 
instruments were drafted, and a detailed work schedule was planned for the 
subsequent years. 

1962-63 -- Pilot Study 

The major activity of 1962-63 was the planijiing and execution of a six- 
week pilot run of the study. It was restricted to a sample of six schools (one 
later had to be dropped) and to the unit test on Sulfur-Nitrogen only, but 
otherwise followed in complete detail all the anticipated procedures of the full 
^tudy. Thus, pretesting with the control instruments was carried out during 
the first weeks of school; students* past achievement dati^ was gathered as 
needed; during the teaching of the unit on Sulfur-Nitrogen, which took about 
six weeks, the Instructional Process Report Form was completed dally; and 
final Regents Examination grades were collected, data were then analyzed 
during the summer, althou^'the analysis was limited to those aspects which 
needed to be checked to assure the success of the final study. 

There were two important purposes to the pilot run. It was, in the first 
instance, a test of the data gathering and data handling procedures contemplated. 
As a result of pilot run experience, for instance, the pretest schedule was 
changed from two 3-l/2-hour blocks of timeion successive days during the 
first or second week of school to a pretest period of 3-1/2 hours In September 

and a second period of like duration in late October. The use of two extended 

I 
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blocks of time ea^iy in the school year, when students are eager to get down 
to work, proved detrimental to morale, and possibly led to dubious responses 
because of that factor, in addition to producing test weariness from seven hours 
of testing in two days. Hence we decided to administer a number of the control 
measures, like IQ, later in the school year at a time when the youngsters 
enjoyed a diversion from the routines of the school day. This change, of course, 
had no bearing on the validity of the study since the measures administered at-^ 
this time were no way influenced l5y the two months of instruction* Also, .during 
this period of try-out, important revisions had to be made in the form used to 
Collect data on students’ prior achievement since guidance information, 
apparently, is maintained in many diverse ways and pan be collected eaeily 
only when adequately planned for. 

The second purpose of the pilot run was to check data processing 
procedures, such as the computer programs, which had to handle the large 
amount of data, and to explore whether each of the planned control measures 
Independently influenced the final outcomes* The conclusion of the latter small 
investigation, which had to be carried out very quickly during the summer just 
preceding the final study, was the somewhat abbreviated design mentioned above. 

X962-63 — Instrument Construction 

Also during the second year of the study, the remaining instrument had to 
be developed and/or put into final form. Thus, coincident with the pilot 
experiihent, the unit tests on Periodic Table, on Water, Solutions, and Ionization, 
and on Principles of Reaction were administered, analyzed, and evaluated. 
The Instructional Process Report Form went throu^ another trial and revision 
early in the academic year. The format that resulted from this revision was 
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then used in the pilot study and proved satisfactory for the full stody without 
further changes. The cost analysis procedures were explored v/ith the aid of 
Professor M, Pullen, who served as consulttot on this part, and a preliminary 
analysis of cost data was completed. Finally, detailed schedule for 1963-64 
was worked out after consultation with the oflcials of each of the schools. It 
incorporated the featpres which resulted from the pilot study experience, as 
well as the limitations and concerns required by the location of the study 
within operating high schools. The final effort of the year was the writing of a 
small, one-page description of the study to explain its nature to the youngsters 
who were to become its subjects, to their parents, and to their other teachers, 

I 

1963-64 — Testing Sessions 

The hill study got under way almost on the first day of instruction in 

-> 

Heptem^r 1963 with the Chemistry Achievement Pretest, a 15-minute test, 
adhiiiiistered by the collaborating teachers before any instruction had taken 
plabe. The other instruments in the pretest battery followed soon thereafter, 
A battery consisting of Science Reasoning and Understanding Test (50 min,), 
Parts I, II, and HI of the Iowa Chemistry Aptitude Test (40 mim), and the 
Kuder Preference Record, scheduled for 60 minutes but for which time was 
allowed until the students completed the entire test, was administered by the 
study staff i^a . 3-T/2-hour testing period sometime during the week of 
_^Septeih6er 9, 1963. Other control data were obtained during a second 3-1/2- 
hour testing session administered by the study staff during the week of 
October 28, 1963, This second set of tests consisted of the Holzinger- Crowder 
Unifactor Tests (90 min,) and the Critical Thinking Test (55 min.). The 
information form which recorded certain personal data as well as the students* 
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past course achievement and the parents* education and occupations was 
inserted betweeh these two tests (15 min.)* The students were allowed excess 
time to complete the /Critical Thinking Test, One control and two criterion 
measures were obtained in a third 3-1/2-hour testing session, late in the 
school year, at a time when all teachers had begun the customary pre-Regents 
Examination review. In the period between May 21 and 29, the study stif 
administered the Brown-Holtzman Survey of Study Habits and Attitudes (20 
min.) and readministered the Science Reasoning and Understandir^ Test and 
the Kuder Preference Rbcord. 

All of the tests administered during the test sessions were scored by the 
study staff and results were furnished to the guidance officios of the individual 
schools as well as to the teachers. The final criterion^strument, the Regents 
Chemistry Examination of 1964, was scored by, the individual teachers and the 
scores were obtained from them. 

' 1903-64 — Unit Tests ' 

Each of the unit tests was administered by the individual teachers at the 
( • 

time when they had cdmpleted work on that particular unit. Since teacshers 
spent various times on the materials covered on each of the unit tests, this led 
to a rather diverse schedule. However, since the staff attempted to measure 
outcomes of the kinds of courses which are typically taught by chemistry 
teachers, no restrictions were put on time, or any other factors related to the 
15 teachers’ Individual teaching plans. Table III shows the wide variation in 
the time for completion of the relevafit units. Indeed, at least one school 
Inverted the order of presenting some of the units^ 

Teachers received keys to the unit tests and scored them for- their own 
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records. They were rescored by the study staff because for the purposes of 
the study, breakdown of items by item type was needed. Each of the teachers 
was ^so furnished with normative information on the unit tests from the 
accumulated data to translate their test scores into percentages. All teachers 
used the unit test grades as a significant part of the course grade so that 
motivation to do well on the tests was kept high. By and large, teachers ‘*liked” 
the tests and found no difficulty in using them as major evaluation devices. 
Both students and teachers recognized the presence of questions that required 
cognitive competencies othef than memorization for the tests through comments 
that referred to them as “reasoning^’ or “critical thinking” tests. That was, 
of course, one of the aims in their construction. 

The test on Principles of Beactipn was an exception to the general plan to 
administer the unit tests upon completion of each unit. Because of an earlier 
misunderstanding, or because changes had Occurred in teaching plans between 
the planning year and 1963-64, it was discovered that the participating teachers 
typically do not teach a unit on principles of reaction but incorporate the 
contents of that unit into their teaching programs as the^opportunity arises. 
Thus, it was impossible to find a block of content of some weeks duration after 
which that unit test would ideally fit, nor, as mentioned above, was it desirable 
to require changes in the teaching plans to establish such a block of content. 
Consequently, the participating teachers were directed to teach their normal 
courses and to administer the Principles of Reaction Test whenever they felt 
that they had adequately taught its contents. As can be seen from the test 
administration schedule which was actually followed, there was some 
homogeneity in the administration of that unit test also. 

Teachers knew the content of each unit test from the tables^ of specification. 



and were sent the actual test booklets at least one week in advance of the day 
for which they r^uired the test in their normal teaching program. They were 
asked not to review unusually for these unit tests— -that is, not to ‘*teach for 
the tests.** 

Instructional Process Report Form 

Teachers were instructed to complete the' Instructional Process Report 
Form from the first day of teaching until the last day of teaching for out-of- 
class contacts related to their chemistry courses as well as for thejlr normal 
assigned classes. In order to reduce some\^at the total burden of reporting, 
a sampling procedure was established for class contacts by requesting a 
report for one section of Regents chemistry only in the conventionally organized 
schools, although all of the participating teachers* sections Wre part of the 
study. The* large group -small group schools reported for the scheduled 
lecture, discussion, and laboratory meetings of one student only, which 
produced the equivalent of ^*one section.** Since alllxit one school scheduled 
Regents chemistry for six or seven periods per week, the sampling procedure 
resulted in at least one and sometimes two reports for scheduled classes each 
day. With reports for the non-scheduled contacts, the average number of 
reports per teacher per day rose to about three. 

Sample Characteristics 
«• , 

% 

Tables IV and V list the schools and give some of the characteristics of 
the instructional groups which were the sample for this study during November 
1963. (Some changes in enrollment occurred during the school year.) The 
conventional classes all met for five class periods per week and had one or 
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two addmonal laboratory periods. Class length was typically 45^47 minutes, 

with one school each with 42 and 52 minute periods* Class sizes varied quite 

considerably, both between and within schools, from a low of 12 to a high of 
31 pupils. 

Variation was greater among the large group -small group schools. One 
school was exclusively large group, five periods per week, with small groups 
meeting only for laboratory. Among the remaining schools two met for two 
large group lectures each week, two for three large group meetings, and one 
^ for four. One or two small group meetip|s were typical although one school 
scheduled three such small- . groups each week. Two of the schools held two 
laboratory sessions each week, four only 6ne. The size of the large group 
varied from 78 to^ 153 and the length of the lecture period from 45 to 50 
minutes. The size of the small groups also varied considerably, from an 
atypical low of 7 to a high of 32. Thirty percent of the small groups contained 

20 or more students which raises some questions concerning their “small 
group” character. 

It should be noted that the teachers in the schools with apparently very 
low enrollments had other instructional assignments and that an additional 
teacher helped with some small group work in the two largest schools. 

While the study sampled schools which had chosen one of the tyo 
organizational patterns by criteria external to the study and were ^us 
separ.ated by whatmightbe termed a receptivity to innovation in the organ^ation 
of instruction, the schools were not otherwise easily distinguisha^. Both 
groups contained large and small schools; both contained schools in urban, 
suburban, small town, and, rural settings. The schools appear similar in 
environmental and, presumably, school-climatic factors. They appear drawn 
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from equally stratified populations, 

I I--. 

Teacher factors .-likewise did not introduce any obvious skewness. All 
, teachers had extensive experience in teaching the Regents chemistry, course, 
and their students had established good records on past Eegbnts Chemistry 
Examinations, 

Finally, the ptudents in the two groups were distinguishable to a 
statistically significant extent only on two of the eight measures'used in the 
analysis of covariance (Table Vjl), and both the significant and the non-significant ; 
differences in mean scores favored one group inhalf the cases and the other fn 
the other half. Hence, no systematic distinction between students could be 
identified. The two findings of significant differences are difficult to interpret. 
At least one is sufficiently large (0.01 level) to be unlikely as a result of chance, 

O 

Nevertheless, even urdikely events happen at times, and this difference might 
represent such an unlikely event. Any effect it may have had on outcomes was 
corrected for by the statistical techniques used in the analysis of Ihe data. 
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Chapter 4 

PUPIL ACHIEVEMENT 



Differences in the effectiveness of- the two organizational patterns for 
achieving learnings of various sorts were evaluated by means ofl5 different 
measures* 

a« The Regents Examination, Parts I and 11, and the Regents Examination 
total score; . , 

b. Six units tests; 

c» Items of all unit tests sorted according to four different intellectual 
behaviors required foi/ their solution; recall, comprehension, 
application, and higher order intellectual behaviors; 

d, Kuder Preference Record, Science Seale; and 

e Test of Science Reasoning and Understanding, 

In order to take account, statistically, of initial differences among the students, 
analysis of covariance procedures were used, with eight independent measures 
as the control variables; 

a,. The Kuder Preference Record, Science Scale, taken as a pretest; 

b The Test on Science Reasoning and Understanding, taken as a pretest; 

Co The Chemistry Achievement Pretest; 

d. The Holzinger-Crowder Unifactor Tests* Verbal, Spatial, Numerical, 
and Reasoning; and 

e. The Pupils’- Scholastic Average for all academic subjects taken in 
9th and later grades up to but not including courses taken during the 
year of the study, 1963~64„ 

These analyses determined differences in achievement under the two 
organizational plans without specific reference to the kind of learner involved. 



They compare mean scores adjusted for differences between pupils on control 
measures. 

Control measures make a maximum independent contribution the analysis 
if the correlation coefficients between them are small. Table VII gives the 
intercorrelations between the eight control variables. Except for the 
intercorrelations on the four scales of the Holainger-Crowder test, the 
correlation coefficients were sufficiently low to justify their use as independent 
control measures. 

Covariance procedures do not permit the assessment of differences in 
the effect of the organi2;ational pattern on pupils with different competencies. 
To ascertain whether bright or slow students fared better under one plan or 
the other, regression analysis was used. This technique derives mathematical 
relationships between the measures that describe the students and the measures 
that determine their achievement*. The coefficients of these mathematical 
relationships are then compared for signifiCiint differences, and if such 
differences exist, the nature of the effect can be investigated. 

The sample of pupils to which these statistical procedures was applied 
was drastically reduced from the total number of pupils in the study since 
each pupil whose data were used in the final analysis had to have taken every 
test. Hence, pupils transferring out of the class, or into the class, or pupils 
who were ill or truant during any of the testing sessions were necessarily 
eliminated from the comparisons. Both the analysis of covariance and the 
regression analysis, therefore, was based on data with '‘healthy non- truants.’* 
While these students generally differed significantly in achiei^ement measures 
from the total group who took the examination, we do not believe that this 
factor invalidates the results of the analysis* This particular problem is 



discussed in detail below. 

Achie vement The Regents Examination . • 

Tables Vin -a-c give the results of the analysis of covariance using the 
Regents Examination as the measure of course achievement. Part I of the 
Regents Examination is a 60-point objective test with items selected from all 
parts of the Regents Syllabus. Usually recall is heavily represented among 
these items t however, in recent years there has been a change in the 
examination toward items requiring other intellectual competencies. The report 
of pupils and teachers regarding the 1964 Regents Examination was that it was 
a “harder^’ ^*thinking-type»’ examinationj that is, it probably contained a 
noticeable fraction of comprehension, application, analysis; synthesis, or 
evaluation-type items although the construction of the test does hot follow a 
design which would assure a desired distribution of different kinds of items. 
Part n of the Regents Examination consists of fewer, more extensive questions 

with a choice among the total number of items. The scoring of Part II is 
objective also. 

For boys, there are a small, though apparently non-chance difference in 
the scores on Part I of the Regents Examination and larger differences on 
art II and on the total Regents score. Both are in favor of the group that 
received instruction under the large group pattern. For girls, the differences 
are not large enough to be statistically significant^ however, they are in the 
sfime direction as for boys. The differences . are maintained for all students 
compared regardless of sex and 'are large enoug6 to be statistically significant. 
Hence, based on our data and within the limitations of our sample and design, 

^ under the large group -small group organizational 
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educationally defensible if higher achievement in. the Regents 
Examination is the desired criterion. /" 

The data also confirm the generally accepted notion that boys perform 
better than girls in chemistry. Boys’ achievement was substantially and 
significantly higher on Part I of the Regents Examination, and this difference 
carried over to the Total Score of the Regents Examination. On Part n, the 
sex difference was not statistically significant. This finding requires further 
comment later since its implications are contrary to others found in this study. 



' Achievement ~~ The 0nit Tests 

Tables DC a~f give the results of the comparisons of achievement scores 
on the six unit tests constructed for this study. These tests are particularly 
appropriate as measuring instruments since their items followed the tables of 
specifications of the teabhers who participated in the study. They reflected the 
teaching sequence of the teachers in greater detail than the Regents Chemistry 
Syllabus, On every test and for every comparison- between large group and 
conventially-taught students, there^is a significant difference in the adjusted 
means of the achievement scores in favor of the large group students. The 
differences approach one half standard deviation, leading to F- ratios of such 
magnitude that a treatment effect seems beyond question. Again within the 
® limitations of the present study, the achievement on each of the six unit tests 
that a decision to instruct under the large group pattern is educationally 
defensible . 

It is interesting that as with the results on the Regents Chemistry 
B,xamination, only some of the unit tests show significant differences between 
males and females. On one unit test the difference is in favor of boys at the 
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0.01 lev^l of significance; two tests produced significant differences at the 
0.05 level; and three tests show no statistically significsmt differences between 
the sexes. 

Achievement — - Intellectual Competencies 

’^en the items of all six unit tests are grouped according to the 
intellectual competencies required for their solution, the same results are 
obtained as when the unit test items are grouped by topic. As Tables X a-d 
show, the differences are very large and in favor of large group instructed 
pupils on all four types of items— those requiring recall, comprehension, 
application, and higher competencies — whether comparisons involve males only, 
females only, or both sexes combined. As in the comparison with the unit test 
items grouped according to topic, the differences approach one half standard 
deviation and strongly suggest the existence of a treatment effect. Again within 
the limitations of the current study, a debisionto instruct under the large group 
pattern is educationally defensible whether the objective is the learning of 
recall, comprehension, application, or analysis, syntliesis and evaluation. 

It is of extreme interest that when the comparisons are made by sex 
alone regardless of the organization of the instruction, there are no significant 
differences on recall and comprehension items but significant differences in 
favor of males on application and higher competence items. These data might 
be interpreted - as a male., superiority on those intellectual competencies 
specifically needed for the problem solving tasks that are integral to chemistry 

\ A 

and which are rated “higher’’^ in -the hierarchy of intellectual behaviors. We 
feel that such an inference would' be rash. We accept as Incontrovertible the 
data that shpw superior male achievement; however, the observed differences 
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are probably caused by factors other than a male superiority on complex 
intellectual behaviors with no difference between the sexes on simpler 
intellectual tasks. We reach this conclusion for at least two reasons. The data 
on the Regents Examination were* exactly converse to those obtained here. On 
the Regents Examination, boys excelled on Part I, the more rote part, and did 
not achieve significantly higher scores than girls on Part II? the problem solving 
part. Moreover/ other data that arose fro=m this study strongly imply that the 
separate intellectual competencies that emerged from our categorizing scheme 
are not psychologically different from each other. As we discuss below, their 
differences seem to reside in a priori logic, not in empirical fact. 

Achievement — Science Reasoning 

In order to determine whether any of the gains achieved in the chemistry 
course could be considered transferrable rather than specific for the syllabus 
content, the Test of Science Reasoning and Understanding was inf^uded among 
the measures of achievement. It could be assumed that any differences on' this 
instrument indicate changes, as a result of instruction, in, the ability to apply 
intellectual behaviors in areas relevant and related to the chemistry course, 
a mlBimum psychological distance from the content of the course, yet not 
specifically related to chemistry. Unfortunately, the Test of Science Reasoning 
and Understanding is only vaguely defined and is no longer generally available. 
Generalized science reasoning is probably a fictitious intellectual category 

V - ' . Q 

(Guilford), and there are few good measurement instruments that remain from 
the time when its existence was postulated without question. Thus in the time 
since the initial' planning of tiiis studfy, evidence has come to light which throws 
some doubts on the interpretation of data oh this test. 



I able XI gives the results of the covanance analysis with the Test of 
Science Reasoning and Understanding. No significant differences can be found 
on any of the comparisons* Neither method of organizing the instruction differed 
in a statistically significant way from the other in inducing generalized reasoning 
with respect to science problems* This does not imply that the year of 
instruction had no effect, A comparison between n^ei&i scores obtained in 
September and in May (Table XII) clearly shows substantial gain.?^ for both 
groups* But the groups were not affected differentiaKy when corrections were 
made for initial differences. 

The data on the Test of Science Reasoning and Understanding require 
additional consideration. The analysis of covariance procedure used the same 
measure to correct outcomes -for initial differences among the pupils. While 
the correlation coefficient's between pretest and' posttest were small, 0»30 and 
0,39 for conventional and large group pupils respectively, the analytical 

s ■ 

technique is extremely sensitive and requires very large treatment effects 
before statistical significance would appear. Perhaps the analytical procedure 
obscures a real effect. This speculation receives some support from the raw 
data used without covariance adjustment which, however, suffer from the defect 
that the students who took the test in September differed somewhat from the 
students who took the test in May because of in-mf^ation, out-migration, and 
absences on the ^wo testing days. If the imcorrected data for all pupils are 
representative of the actual results of instruction under the two organizational 
patterns, then science reasoning is the only measure in this study in which 
conventionally taught pupils achieved significantly higher scores at the end of 
instruction than large group - small group taught pupils. 

The various ways in which our data can be interpreted suggest that the 



pattern, tiie ‘‘heal&y non-tmant^’ those pupils for whom we have data on 
every test administered, are not a random sample of the total students in the 
study. Tables XV a,b show that for the large group taught studepts on every 
measure except the kuder Science Scale, and for the conventially tau^t 
students on about two-thirds of the measures, the restricted, covariance 

^ t 

analysis sample differed significantly from all the shidentp who took the same 

i 

tests. It suggests that studejnts v^o are more diligent in their, attendance 
achieve more hi^ly, or that j students who are migrant or ill are penalized in 
their achievement. That finding is not surprising, and since covariance analysis 
adjusts the data to offset problem^ arising from , this source, it does not 
invalidate the conclusions we have dra^, regarding achievement differences 
among the two kinds of teaching. But these data led, us to examine the results 
Obtained on each test fpr all students and to compare tlie data to see whether 
we could get any additional insists into just exactly what was happening to 
different students under the two organizational patterns! ihs fact that the 
initial samples appeared indistinguishable, with each group excelling on half 
the control measures and “significantly higher on one, gave some justification 
for this unplanned analysis. 

The effects found with the analysis of covariance on the restricted sample 
carry over to the total sample (Table XVI). The large group -small group 
students surpassed coiv^entionally tau^t students on 13 of 15 comparison 
measures though differences were statistically significant on only 9 of 
them. Differences were significant at tiie 0,01 level on three unit tests, on 
Type in items, and on Part I of the Begents Examination. Significance dropped 
to the 0,05 level for another unit teat. Type IV items, and Part H and total 
score of the Regents Examination. On the other hand, the conventional students 



64 



0 



f ^ . .. , 

■ <3 r 



pattern, tiie “heal&y non-truant^’ those pupils for whom we have data on 
every test administered, are not a random sample of the total students in the 
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their achievement. That finding is not surprising, and since covariance analysis 
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Obtained on each test fpr' all students and to compare tlie data to see whether 
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different students under the two organizational patterns! The fact that the 
initial samples appeared indistinguishable, with each group excelling on half 
the control measures and “significantly higher on one, gav/s some justification 
for this unplanned analysis. 

The effects found with the analysis of covariance on the restricted sample 
carry over to the total sample (Table XVI). The large group -small group 
students surpassed coiv^entionally tau^t students on 13 of 15 comparison 
measures though differences were statistically significant on only 9 of 
them. Differences were significant at tiie 0,01 level on three unit tests, on 
Type in items, and on Part I of the Begents Examination. Significance dropped 
to the 0,05 level for another unit test, Type IV items, and Part n and total 
score of the Regents Examination. On the other hand, the conventional students 
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(N for Covariance Sample is 344) 
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(N for Covariance Sample is 344) 
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scored higher on the Test of Science Reasoning and Understanding at the d.Ol 

■ ■ ^ 

level of significance, and non-significantlv on the Sulfur-Nitrogen ^f;est. 

The uncorrected data generally support our initial conclusions regarding 
higher achievement in the large ^oup- small group schools. But it is difficult 
to see from the data just what is happening. The restricted large group sample 
was somewhat more select than the restricted conventional sample and it also 
was relatively smaller. Only 64% ofthe large group pupils who took the Regents 
Examination were in the restricted cample in contrast to 75% of the conventional 
pupils. Attendance was apparently better in conventional classes. Perhaps 
more importantly, persistence in the cpursd was also greater in the conventional 
classrooms (Table XVII) , The net decrease in enrollment from September to 
June, as determined by the number of students who took our tests, w?b 10% for 
conventional classes whereas the^ net decrease was 17% for the large group - 
small group dlasses. While these figures need hxrther study since in-migration, 
out-migration, and illness affect them and since counselors might hesitate to 
place transfer students into large groups in mid-term, we feel that a significant 
residual effect probably remains ^whidh simply drives "^^larger fraction of 
students from large* group taught chemistry classed. The relative losses 
between October,. by which time classes typically have stabilized, and June are 

^ .. - 5 

2.3% and 10.8% respectively. The data signify, we believe, that students withdraw 
from large group - small group classes in significantly greater numbers than 
from c onventional classes. 

■ ' ' ’ ■ ■■• ' • ■■■■ ■■ VI - J - I . ■ - ■ — - - - - « . 

The last few paragraphs have thrown some light on a number of factors 
which ar^ of great educational significance and whichdeserve to be investigated 
more fully. Quite obvious^VrJBven thd exhaustive analysis of achievement 
measures which was undertaken in this study does not fully answer all questions 
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achievement of non-specific science-related intellectual behavior, as 
represented by the Test of Science Reasoning and Understanding, is an area in 
which additional research would be fruitful. The preliminary conclusions, in 

«> i‘ 

^ view of the design of our study, is that neither group was affected differently 
from the other in the competencies measured by the Test of Science Reasoning 
and Understanding, 

Achievement — Science Interest 

Motivation plays an important role in the advantages cited by proponents 
of the large group organization in the high school. Unfortunately, it is one of 
the most difficult t^i^ings to measure. The best me^ure would be performance 
in later years--college preparation for and actual persistence into a science- 

... ^ i ■ 

related occupation— but data could not be attained for several years. (We are 
prepared for such, a follow-up study in future years.) As an approximate 
indication of motivation toward science we chose'the Kuder Preference Record, 
Science Scale. It was administered both before the study, to differentiate 
pupils, and after the study, to assess changes in outcome. As v/e" indicated 
earlier; the two groups appeared undifferentiated on this measure before the 
study began, ■ . . 

‘ The treatment apparently had little or no effect on science interest. The 
correlation between scores "measured in September and in May was near 0.65 

for all groups and sub-groups, showing a significant influence of initial interest 

/ 

on the interest score at the end of instruction,^ Moreover, there were only 
small .-gains on the science interest scores between pretest and posttest 
(Table XIII) for the uncorrected samples. The result persisted in the covariance 
analysis (Table 3a V). ’ 



59 






i' 








a 

S 

w 

a 



e-t 



m 

a 

< 

o 

CQ 

w 

Q ^ 

m y 

O 

02 g 

dg 

OT 

O 12 
o ^ 

^§' 
M ® r■/^ 

“!-S2 

w w ^ 

fa JU w 
fa E Q 

s 

fa ei B 
^ fa w 

fa ^ ^ 
fa fa 

9<d 

fa Q 



^ fa 



fa 
fa ^ 

o 5 

fa,o 

fa 

Jr fa 

*~i 

fa fa 

■^2 

tZ2 



fa • 
O fa 
fa fa 
fa 



fa 



p 



r * fa 

o 

02 



fa 

Q 

fa 

fa 

fa 

W 

fa 

fa 



P 



§5 

^ >*HI 

< fa 

pS 

fa t"* 

< > 

-5 w- 

02 P 



fa 

< 

a 



fa 

fa 

fa 

p 

fa 



a 

fa 



U2 

a> 



t> 

Oif 

d. 



U> 

iti 



CO 

:z; 



a> 



<o 

ID 



0> 

00 

* 

o 



<0 

o 

* 

o 





O 


CO 


0> 




CO 


o 


CO; 


tr- 




C- 


CO 






lO! 


io 


ci 


CO 


to 


09 


to 


r-4.- 


r~i 


r*( 


rH 


,. tH' 


tH 


rH 





<0 


o 


' t- 


<0 




00 




Oi 


to. 




09 


vH 


c4 


1H[ 


r4 


c4 








rH 









SO 


T»t 






to 






lO 


C- 


H 


<?> 


to 


f-4 




<D 


^ C0 


lO 


to 


CD 


. a 


10 



>Hf 

H 

fa 

a 

p 

l-H 

fa 

P 

9 

fa 

o 



■ fa 
0) 
a 

! 

a 

a 

o 

D 

0 

!■') 

cd 

a 



4o> 

fa 

a) 

a 



tsJ 

« 

,2 

i 

c 

o 

□ 





s> 


« 












t 


1 


1 


1 


s* 

o 


a 


fa 

g 


04 

o 

o 




2 


u 




a 


t 


O 


O 


<u 


0 


0) 


Q> 


h'^ 


>> 


h-t 






C3 






a 


vj 

Q 


a 


a 



ts' 

0t< 

0> 

CO 

I 

l*-la 

CJj 

a 

o 

*r^ 

d" 

0 

1 

Q 



60 



ConventiOAial - May 591 41,90 13.79 
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. One difference, however, stands out from the covariance analysis. There 
is a significant difference between boys and girls in science interest at the end 
of the year despite the fact that thei^r scores were adjusted for initial differences 
on the same measure. During the year they studied chemistry, whether by 
conventional or large group - small group instruction, girls became less 
interested in science while boys became more interested, at least as far as 
this interest is measured by the Kuder Preference Record. This finding has 
important implications for occupational guidance and considerations of 
manpower need and'aupply. 

The Scien ce Scale of the Kuder Preference Record shows no differences 

_ . ^ M . n ,,.i. I, p ■ ■ 

between cbnventional and lar^ group taught pupils, no gains of interest when 
sexes , are not separated^ but a gain for boys and a loss for girls cf statistical 
significance between the September and May. , , 

■ ■■ . 5 

Student Differences . 

In the comparisons we have discussed up to this point, we used data 
without regard-for pupil differences Except for sex. Tp determine whether tvwj 

\ " - a ■ 

organizational patterns affect differeht sub-sahaples of. students differently, 
we used recession analysis. We wbre concerned with whether one or th^ 
other kind of instruction favored students j\^^o scor^ hi^ or low oh any or 
all of the ei^t control variables. 

As mentioned in the beginning of this chapter, regressichi analysis 
establishes mathematical equations that relate the control variables to 
achievement variables. The terms of the resulting equations fpr the two 
treatment groups are then compared and tested for statistically significant 
differences. With 8 predictor variables and 15 criterion variables, 120 slirq)le 



h 




recession equations and more than 1000 multiple regression equations result. 
Since the computer program required a score for every subject on each of the 

C ^ .’J. 

23 measures, this analysis also was limited to the somewhat restricted sample 
of students who were neither ill nor truant during the year. 

The number of recession coefficients produced by the analysis is too 
large to be . reproduced in this .report. All simple and possible multiple 
regression equations were computed and their coefficients compared for the 
large group -small group and conventional students. Among the several 
thousand comparisons, only ahandfol gave differences that appeared statistically 
s^ignificant at the 0.05 per cent level. Since chance alone produces differences 
of that ma^itude in one trial of 20, ^d since the actual data produced fewer 
than that fraction of apparently different regression lines, we attach no 
importance to the differences which appeared significant and ascribe them 
entirely to chance* The fact that no pattern could be detected among the few 
results that seemed significantly different lends further support to that 
inference. We could not identify any subgroup of stjidents, with hfj^ IQ or 
lower IQ, with hig^ner achievement records or lower achievement records, 
with better chemistry preparation or poorer chemistry preparation, or with 
greater or lesser science interest, for whom achievement results differ^ 
from the results obtained with the entire sample, (The only exception, as 
noted previously are male or female subgroups.) Hence, we conclude that 
there are no subgroups of students who are affected differently from the total 
sample by either organisational pattern for instruction . 

, Nevertheless, among the wealth of data accumulated in the course of the 
shidy, there are a number of indications concerning different effects which 

^ require further attention. In the first place, regardless of organizational 

‘ ' ' ■ ■ 
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pattern, the “healthy non-truant^* those pupils for v^om we have data on 
every test administered, are not a random sample of the total students in the 
study. Tables X¥ a,b show that for the large groi^ taught studepts on every 
measure except the Kuder Science Scale, and for the conventially tau^t 



students on about two-thirds of the measures, tiie restricted, covariance 

■f ' t ** 

analysis sample differed sigoifiesmtly from all the student^ who took the same 
tests. It suggests that studejnts who are more diligent in their, attendance 



achieve more hi^ly, or that j students who are migrant or ill are penalized in 
fheif achievement. That findinj^ is not surprising, and since covariarice analysis 



adjusts the data to offset problemf arising from ,this source, it does not 
invalidate the conclusions we have drawn regarding achievement differences 
among the two kinds of teaching. But these data led us to examine the results 
obtained on each test fpr all students and to compare the data to see whether 
we could get any aidditional insists into just exactly what was happening to 
different students under the two organizational pattern^ The fact that the 
initial samples appeared indistinguishable, with each group excelling on half 

the control measures and "significantly higher on one, gave some justification 

■} * 

for this unplanned analysis, ’ 



The effects found with the analysis of covariance on the restricted sample 
carry over to the total sample (Table XVI). The large group - small group 
students surpassed conventionally taught students on 13 of 15 comparison 
measures though differences were statistically significant on only 9 of 
them. Differences were significant at the 0,01 level on three unit tests, on 



Type in items, and on Part I of the Begents Examination. Significance dropped 
to the 0,05 level for another unit test. Type IV items, and Part H and total 
score of the Begents Examination. On the other hand, the conventional students 
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scored higher on the Test of Science Reasoning and Understanding at the.d.Ol 
level of significance, and non-significantlv on the SulRir-Nitrogen ^st. 

The uncorrected data generally support our initial conclusions regarding 
higher achievement in the large ^oup- small group schools. But it is difficult 
to see from the data just what is happening. The restricted large group sample 
was somewhat more select than the restricted conventional sample and it also 
was relatively smaller. Only 64% of the large group pupils who t^)ok the Regents 
Examination were in the restricted sample in contrast to 75% of the conventional 
pupils. Attendance was apparently better in conventional classes. Perhaps 
more importantly, persistence in the c, pur sS was also greater in the conventional 
classrooms (Table XVII), The net decrease in enrollment from September to 
June, as determined by the number of students who took our tests, was 10% for 
conventional classes whereas the' net decrease was 17% for the large group - 
small group Classes. "While these figures need further study since in-migration, 
out-migration, and illness affect them and since counselors might hesitate to 
place transfer , students into large groups in mid-term, we feel that a.^ignificant 
residual effect probably remains^, whidh simply drives '^^iarger fraction of 
students from large* group taught chemistry classed. The relative losses 
between October,. by which time classes typically have stabilized, and June are 

^ • s 

2.3% and 10.8% respectively. The data signify, we believe, that students withdraw 
large group - small group ciasses in significantly greater numbers than 
from conventional ciasses. 

— " - — ■ ■ - 1 ■ - 1 . 11 ..— — ■ - - « ' . 

The last few paragraphs have thrown some light on a number of factors 

which ar^ of great educational significance and whichdeserve to be investigated 
more fully. Quite obvious^^T^ven thC exhaustive analysis of achievement 

* ' c » ■ ' 

me^ures which was undertaken in this study does not fully answer all questions 



TABLE XVII 



NUMBER OF STUDENTS WHO I^OOK TESTS IN SEPTEMBER, 
i . OCTOBER AND JUNE 



DATE AND, TEST CONVENTIONAL LARGE GROUP 



* 


N 


LOSS 


%LOSS • 


N 


LOSS 


%LOSS 


First week of school 
(Ohem» Pretest) 


673 






. 650 






Late October 
(Holzinger-Crowder) 


. 621 


52 


7.7 


609 


41 


6,3 


Late June 

(Regents Examination) 


606 


67 


10,0 


539 


111 


17.1 



SIGNIFICANCE OF DIFFERENCES BETWEEN LOSS RATES 
UNDER THE TWO ORGANIZATIONAL PATTERNS 

i 

I 

i‘ 



DATES COMPARED 


CHI-SQUARE 


LEVEL OF 
SIGNIFICANCE 


First week of school 
and late October 


0.8 


NS 


Late June and first 


week of school 


15.0 


0.01 


Late June and late 
October 

y 


73.5 


0.01 
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one must raise with regard to large group - small grioup instruction. Other 
values enter the picture. We now know that the holding power of the large group 
classes is poorer. We have some vague intimations that it may work 
preferentially against poorer students although our data are insufficient to 
clear up this point. But we do not know how poor these drop-outs are. Are 
they so poor that they would hardly have profited from persfstericef or were 
they poor but generally successful stddents? Further data would help answer 
some of these vital questions. 

Classification of Intellectual Behaviors 

Several times already w© have cast doubts on inferences with regard to ’ 
different intellectual competencies although in several places in our initial 
research design, we assumed that such competencies exist and can be measured. 

Subsequent experimental results, however, have caused a change in our 
assumptions. 

In the first place, there were contradictory results from the Regents 
Examination, on which males surpassed females only on the memory part, and 
from the various levels of intellectual complexity on the unit tests, on which 
males surpassed females on application and higher order competencies but 
not on memory. Secondly, there was the Test on Science Reasoning and 
Understanding which did not corroborate the significant differences in favor of 
large group -small group students found on supposedly comparable higher order 

J ■ 

competency items. Finally, there is the recent work of Guilfo:|:d on the structure 
of creative intellectual processes which shows them to be quite complex, 
more complex than the theoretical stmicture developed by Bloom and used as 
the basis for our differentiations. But, we have direct evidence that our 



adaptation of Bloom’s categories has no empirical basis. Intercorrelation 
coefficients, between the items of Types I, II, HI, and IV, for both conventional 
and large group pupils, are not less than .80 (Table XVIII). Hence, the items 
reach nearly the limit of reliability of any set of related items and 
differentiations among them ifardly-^,emst. Correlations between the items of 
various types and other measures, likewise, do not distinguish among them. 
The correlation coefficients are almost identical , for items of Type I, II, IH, 
and IV and each of the control measures used in Ms study (Table XIX). 
Indeed, it seems particularly significant that the correlation between the 
Holzinger-Crowder Keasoning Score and our item Type IV is non-significantly 
larger than the correlation between that score and our item Type I. 

There can be no question that our items do not empirically differentiate 
the presumed intellectual behavior categories adapted from Bloom’s taxonomy. 
There are several different interpretations possible, however. It may be that 
our items were incompetently consi^ructed for the purposes at hand. This is 

* . , i • c . « 

possible, though a jury of X4 teachers agreed with little difficulty to distinctions 
between recall and the other three item types (if not as easily distinctions 
among the other item types). Yet, we find no empirical distinction even between 
items of Type I and the other types. Another explanation suggests that there is 
a general factor operative among all items in addition to the distinguishing 
mental competencies of the more complex types. Recall is admittedly required 
for all items and it may affect them to an overriding extent, or some other 
gene-al factor may affect them to an overriding extent. This is quite likely. 
However, it makes the categories operationally indistinguishable and therefore 
theoretically useless. Whatever the reason for the hi^ intercorrelations, we 
feel that our data demonstrate very clearly that the categories we used are 
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T7VBLE XVni 

INTERCORRELATION MATRIX BETWEEN RECALL, COMPREHENSION, 
APPLICATION, AND HIGHER ORDER COMPETENCY ITEMS 



n HI ■ IV 
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Item Type I 
Recall 

Item Type II 
Comprehension 

Item I^e HI 
^Application 

Item Type IV 
Higher Order 
Competencies 



.87 .84 .81 



.84 .80 



,82. 



h 







i 



i 









"i ^ W.' 



J., M ,, INiy 41 ■' 1. ft up.. m • » 







o 



ERIC 



Bi’lliitlilBIffTliLiU 



*.3 



,-X.. 



\ 



TABLE XIX 



COREELATION COEFFICIENTS BETWEEN ITEM TYPES 

AND OTHER MEASURES 



* ITEM TYPE ITEM TYPE 

OTHER MEASURES LARGE GROUP PUPILS CONVENTIONAL PUPILS 





I II HI IV 


I 


H in 


IV 


Chemistry Regents 


.64 .69 .70 ..60 


.75 


.77 .74 


-.71 


Scholastic Average 


P 

.49 ,51 ,55 . ,50 


.54 


.56 ,55 


.56 


Kuder Science, 
Pretest 


.09 .10 .10 .09 


.25 


.23 .22 


.24 


Chemistry Pretest 


.37 .35 .39 .40 


.52 


,55 .53 


.52 


Science Reasoning 
Pretest 


.21 ,24 .28 .26 


,37 


.36 .39 


,40 


Holzinger-Crowder 

Verbal 


A 

.27 .27 .31 .28 


,47 ' 


.48 .43 


.45 


Spatial 


.18 .22 .22 .19 


.32 


.36 .33 


.32 


Numerical 


.29 .29 .32 .27 


.39 


.41 .42 


,36 


Reasoning 


.22 .26 .31 .27 


.37 


,42 .44 


,42 



N for Large, Group * 609 or more 
N for Conventional — 621 or more 
Type I . - Recall 
Type m *- Application 



Type H - Gompreliension 

Type IV - Higher Intellectual Behaviors 
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empirically indistinguishable and that their use in learning and curriculum 
theory pl^es a psychologically non-existent phantom into the discussions. We 
would suggest that there are, no doubt, intellectual competencies of various 
orders but that the words recall, comprehension, application, analysis, 
synthesis, and evaluation, are inadequate to describe them. This too is an area 
in v^ich much valuable research remains to be done. 
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Chapter 5 

PREDICTING ACHIEVEMENT 

The equations relating pretest scores with achievement scores showed no 
difference between large group -smaii group pupils and conventional pupils. 
But, the correlation coefficients were high in many cases. This suggested the 
.possibility of making a fairly reliable prediction of Regents Examination scores 
early in the school year from one or two test scores. Such prediction, of 

* 

course, assumes that the difficulty of the Chemistry Regents Examination will 
remain stable. Since the 1904 examination was Judged to be relatively difficult, 
such an assumption is probably justified although if the difficulty level should 
drop, prediction based on the 1963-64 data would tend to underestimate 
" achievement. That could be considfered an advantage since a final score jhi^er 
than predicted may be considered more desirable in practice than the reverse. ' 
If the Regents Examination should become more difficult, on the other hand, 
the use of the data of this chapter would probably be unjustified. 

Anyone \mo attempts to make a prediction about individual behavior is 
dealing with tenuous probabilities, and this is perhaps more true for a prediction 
of a score on a single examination that for a prediction of overall success in 
hi^ school* Nonetheless, teachers, counselors, and even pupils and their 
parents have always made predictions about future performance, and they will 
no doubt continue to do so. The very decision to enroll in a course like Regents 
Chemistry vmich is elective and which is known to be intellectually demanding 
involves an » elefhent of prediction: the supposition that the pupil will be 
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successful in meeting the*" course requirements. The puiTpose of statistical 
.prediction, then, is to objectify prediction and, hopefully, to make it more 
precise and 'accurate than the subjective implicit predictions uponvMch 
important decisidns are generally founded. / 

Making predictions about the future performance of groups of students is 
much safer, of course, than making predictions about individuals. In group 
predictions, substantial variations on the part of individuals balance one 
another, and the mean predicted score will be quite close the actual mean 
achieved score. But systematic errors will still be reflected in the outcomes, 
so that differences in the difficulty level of the criterion test, differences in 
the students with respect to fach>rs that are not considered in making the 
predictions but still exert an influence upon the criterion outcome, and 
differences that may not o^erwise be identifiable all contribute to error. 

In considering thd results that are presented in this chapter, we must 
remind the reader that he is viewing the results associated with students v^o 
were ^rolled in Regents Chemistry during the 1963-64 school year, and that 
the results apply, directly, only to die Regents Examination adimiriistered in 
June of that year. The accuracy of predictions based upon this experience, 
therefore, will be directly related to the congiuence between the characteristics 
of pupils enroll^ in chemistry during that year and in subsequent years, and 
to the equivalence of Regents examinations administered that year and in 
subsequent years. 

Simple Correlation 

Table XX/^ves the equations for the regression line connecting 
achievement scores on the Chemistry Pretest and each of the six unit tests 



with the scores on the Regents Examination. We note that the correlation 
coefficient is substantial, between the Pretest and the Regents Exainination 
* and that the correlation coefficient increases markedly for the test on the 

W 

first unit of instruction, Periodic Table and then increases more gradually for 
« the testa given later in the school year. The standard error of estimate changes 
accordingly; the standard error of estimate from the Chemistry Pretest is 
substantially larger than the subsequent ones, and those decrease as the date of 
test administration approaches the date of the Regents Examination. 

The information contained in Table XX can be given in another form 
which is easier to interpret, tDne can take the score achieved by a pupil on 
the Chemistry Pretest or on one of the unit tests and estimate his most likely 
score on the Regents Examination. This is done in Table XXI. 

One enters Table XXI with the score the pupil achieved on one of the 
tests in the cohimn headed by the name of the test. For instance, if a pupil 
scored 26 on the Chemistry Pretest, one would look in the column headed by 
“CHEMISTOY PRETEST*^ to the entry Then, reading hdrizontally 

across to the column headed “REGENTS EXAMINATION,’’ one obtains an 
estimate of his most likely Regents Examination score, an 80 in this hypothetical 
case. However, this is the most likely score only. Actual scores for pupils 
with a 26 on the Chemistry Pretest distribute themselves about 80. To get an 
estimate of how widely they will tend fo range, we look at the last column entry 

under Chemistry Pretest, the SO per cent range entry. This tells us that 50 

% 

per cent of the impils who score 26 on the Chemistry Pretest have Regents 
Examination grades within 9 units of 80; 50 per cent have Regents Examination 
grades between 71 and 89. This hypothetical pupil then has a probability of .50 
of scoring between 71 and 89 on the Regents Examination, a probability of 



p 



^ ’ f"-. 

* 

, .25 of scoring below 71 on the Kegents Examination,' and a sinailar probability 
of .25 of scoring higher than 89. 

Or, as another example, a student scores 27 on the Halogen test. Reading 
across from the entry In the / “HALOGEN” column predicts 65 as the most 
likely Regents Examination score* For this pupil the data indicate a 50 per 
cent chance of passing that examination. With a 50 per cent range of 7 for the 
distribution of predicted Re^nts Examination, hirthermore, the probability is 
.50 that his score will be between 58 and 72, .25 that it will be below 58, and 
,25 that it will be above 72. In constructing Table XXI, numbers were rounded 
to the nearest vdiole number. For intermediate scores, one must interpolate 
in both columns. 
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Multiple Correlation 

Since approximately half of the total variance on most of the pairs of 

predictor tests appeared to be shared variance, the prospects for improved 

prediction through multivariate analysis seemed favorable. This was born out 

for pairs of predictor variables (Table XXII). The multiple correlation 
* ^ 

coefficients are larger and the standard errors of estimate are reduced. 
Employment of additional predictor variables beyond two, however, did not 
increase correlation , sufficiently to justify the increased computational 
complexity* The table gives the multiple regression equations for successive 
pairs of tests used in this study. As before, a substantial part of the variance 
on the. Regents Examination scores can be attributed to the variances of the 
earlier tests and the common variance increases as the dates of test 
administration approach date when the Regents Examination is given. 
The inform^Htes^=dbntained in the multiple regression equations can also 
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be given in somewhat simpler form. For this purpose, we have constructed 
the nomographs in Figures 1-6. The nomographs are used as fdllowsi 

1. Locate the two test scores to be used in prediction on the appropriate 
coordinate axes. (Interpolation may be necessary.) 

2. Find the point of intersection of these two coordinate in the nomograph, 

3. Interpolate between the diagonal lines to estimate the most likely 
Regents Examination score, (The numbers on the diagonal are the 
most likely Regents Examination scores.) 

For example, a student scores 35 on the Chemistry Pretest and 30 on 
the Periodic Table Test. In Figure 1, the intersection of those coordinate 
lines is very near the diagonal which predicts an 80 on the Regents Examination. I 
However^ we feel we must stress again that 80 is only the most likely score, | 
A group of students with 35 on the Pretest and 30 on the Periodic Table Test 
will obtain Regents Examination scores that distribute themselves around 80 
so that any one student has a good chance of scoring higher or lower than 
80, In order to simpli^ the nomographs, ranges were hot ejqplicitly given. 
They can be computed- from the multiple regression equations. However, all 
pupils whose predicted scores are at least 70 ’ have a high probability of 
passing the course. 

Since one of the chief uses- of the nomographs may be in the early 
identification of pupils who are likely to fail, the three bottom left diagonals 
of the nomographs contain estimates of the probability of passing the course. 
Thus a student with a score of 30 on the Periodic Table Test and a score of 
25 on the Halogen Test (Figure 3) has only a 50 per cent chance of passing 
the Regents Examination; with scores of 20‘onboth tests, his chance is slightly 
less than 10 per cent. This information may be helpful in changing study habits 
or in early recognition of probable failure. 

We would suggest, however, that if the information of this chapter is 

» 
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Nomograph for Regents Examination Scores Associated with 
Pretests and Periodic Table Test Scores • 
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Score on the Periodic Table Test 
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^ore on the Periodic Tjable Test 




Scca:*©^o]n„the I|.alogen Test 



Nomograph for Eegeats Examination Scores Associated with 
Sulfur-Nitrogen and Organic Chemistry Test Scores 
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shown k)' students, that it be used without direct reference to prediction since 
the prediction is accurate only with groups of students, not with any single 
case. It is probably ‘ preferable simply to stress the experiences of other 
students who had similar scores on the predictor tests, and to point out that 
a number of other factors contribute . to the determination of any particular 
student’s Regents Examination score. It would probably be * appropriate also 
to point out that the Regents Examination measures something somewhat 
different from wbat the predictor tests attempted to measure. 

A final word of caution is in order with respect to the possible use of 
these regression data in districts other than those whiclrparticipated in the 
study. To the extent that other districts, their teachers and/or their pupils 
differ from those represented in the study, predictions based on these data 
may be even less accurate than within the participant districts; there is no 
way of telling this without empirical evidence, ^ 
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Chapter 6 

INSTRUCTIONAL PRCXJESSES 

With the large and significant differences on most achievement measles 

between large group -small group students and conventional students, it is 

particularly important to determine whether organizational pattern alone 

produces the different results or whether more specific teacher behaviors^ can 

be found to viiich achievement difference can be attributed* Consequently, we 

carried out several different analyses of the data from the Instructional 

Process Description Instruments in an attempt to discover commonalities 

among the practices of the large group teaphers and of the conventional 

teachers* The details of the organizational pattern^ in the six large group 

♦ 

schools (Table V) differed to such an exteni that the mere existence of large 

> ■ ■ ■ 

groups and of small groups appeared as ihe only common feature among them. 

The first analysis of variance attempted to determine differences in the 
time spent by teachers in the two patterns on each of 23 different teacher, 
pupil, or teacher-pupil interaction behaviors* These were the 23 behavior 
categories of the Instrument (Table XXIII) , On none of the behaviors was there 
a significant difference which could be ascribed to organizational pattern. On 
iiiost, variations within each organizational pattern were a| least as great as 
the difference between large group -small group and conventional teachers. 
No distinct teacher behavior category could be identified which might be 
considered characteristic of one dr the other of the organizational patterns. 
An analysis of variance was also made on the ten objective categories 
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TABLE XXni 
BEHAVIOR CATEGORIES 



t 



«- 




(. 



Teacher does .... ' . 

Teacher talked, while the students listened and/or took notes or recorded 
statements verbatim; illustration, demonstration, student questions, etc., 
were only incidentally employed if at all (I) . 

Teacher used chemistry-laboratory materials and/ or equipment to illustrate 
some principle directly related to a specific objective of the course (1^ . 

Teacher used other means of communication to transmit, demonstrate or 
illustrate facts or principles (e.g„ films, filmstrips, recorders) (I). 

Teacher administered a written h-st, the students having at least one day’s 
notice or expecting the test on that day (II), 

Teacher administered a written test, the students having less than a day’s 
notice (H), 

The teacher discussed with and/or lectured to the class pn some topic 
having only incidental (or no) direct relation to any specific objective of the 
chemistry course (e.g„ segregation, the four-ininute mile, federal aid for 
education) (I), ' 



Teacher and Student , , . 



Teacher directed questions and designated individual students of his own m 

selection to provide the answers orally. Students may or may not have . , 

pre^idously seen questions (e.g„ homework) (III). ^ ' 

Teacher directed questions but restricted his seleqtion of the student to [ 

answer to those indicating their willingness (viz., byraising hand, etc.) (III). ’ 

Students directed questions ‘ to the teacher/ andhe answered them or. elicited ^ 

the answers from other students (IV). ’ • 

Teacher and the students engaged in open interchange of ideas without 
formalized structure (i.e., students did not withhold comment or participation 
until recognized by the teacher). (IV). ^ 



Student . , , 



Students engaged in open interchange of ideas without formalized structure, 
and the teacher’s participation was minimal, or he took no part at all' in the 
discussion (IV). 



o 



I 
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TABLE XXIII (Continued) 



Indivi>daal on small group of students presented a demonstration, using 
ohemistry-laboratory materi£^s and/or equipment while the other shidents 
listened ^d/or tool^dtes 

Individual o^ small group of students presented a report or panel discussion 
while other students listened and/ or took ngtes. The presentation was 
primarily verbal; illustration and demonstration y/ere used only incidentally 
/}if at all (III). ’ . 



Stodents engaged in laboratory work (actual manipulation of lab equipment) 
directed toward tlie fulfillment of a specific course objective. Teacher gave 
normal supervision (V). 



/ 

All students present worked at the blackboard solving.problems assigned by 
the teacher (III). 



Identical with above, except that only a part of the class worked at the 
blackboards the remaining students observing and/or solving problems at 
their seats (ni). . 

, Students solved problems or wrote verbal r^ponses to questions assigned 
by the teacher. Teacher exercised general supervision and gave advice or 
assistance as requested. (TMs may or may not be a head start on homevrark 
assignment.) (HI) / . . 

i 

I ■* ... 

Students studied text or other materials for general learning rather than to 
answer specific questions. Teacher exercised general supervision and gave 
advice whOn requested (VI). 



Students scored their own or another student’s test paper, unde/ teacher’s 
supervision (VII). \ ' . 

Students policed the laboratory or performed other \ tasks more directly 
related to maintenance of the classroom facility than td the achievement of 
specific course objectives (VII). 



Miscellaneous .... 

• 'V 

The teacher engaged in some activity of a primarily procedural nature 
(e.g„ taking roll, '^reading announcements, handing our papers, announcing 
homework or tests) (VII). " 



Interruption in the class (b.g., P-A announcement, fire drill— objective code 
X will usually be appropriate here) (VII). 

Someth 'ng not classifiable uniier any of the’ above categories (VII). 



(Table XXIV) dt the Instrument. These were designed to elicit the purposes of 
the instructional effort. Like on the behavior categories, no distinction was 
found which would divide the 15 teachers clearly into the large group -small 
group or conventional pattern. Time spent ^n various objectives of teaching 
also varied as much within each pattern as between the patterns. 

Since no characteristics could' be identified which clearly distinguished 
all large group teachers from all conventional teachers when teacher behavior 
categories were examined for each teacher individually, we grouped data to 
see whether any distinctive patterns emerge for either organizational pattern. 

r 

Since many of the 23 behavior categories require only negligible fractions of 
the teaching time, this procedure failed to give an insi^tful description of 
the range of teaching behaviors, and the data are not reproduced here. Not 
surprisingly, conventional teachers spent three times as much time in directing 
questions for voluntary pupil response (15% vs. 5^ of the teaching time). 
Also not surprisingly, large group teachers employet^ common audio-visual 

aids— films, filmstrips, etc.— twice as much as conventional teachers (8% vs, 

\ 

4% of the teaching time). But perhaps surprisingly, (^emonstration-lecture 
time differed only slightly between the two patterns, with conventional teachers 
spending 27% of their time on this activity contrasted with 32% of time for 
large group teachers. Our initial hypothesis that the details of teacher behavior 
may differ less between the two patterns and that class size may be the major^ 

operational variable seems to be supported. 

< 

An expected pattern emerges when the 23 behavior categories are further 
grouped into groups of related interactions. By combining the categories as 
indicated by the Roman numerals in Table XXIII, we arrived at seven groups of 
behaviors; teacher dominance (I); evaluation (II); teacher initiated interaction 
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TABLE XXIV 



OBJECTIVE CATEGORIES 



1. To introduce and/or develop a concept relevant to the chemistry courses. 
This may involve specific facts, generalizations, reasoning procedures, 
definitions, names of equipment or descriptions of how equipment operates. 
This category should be used when the primary reason for carrying out 
the activity is the initial development of a new concept, even though 
references may be made to concepts previously developed in order to 
further the development of the new concept. 

2, To correct an erroneous concept acquired by one or more students in the 
chemistry class, in some other class, outside of classes, or prior to 
beginning i^e coui^se in chemistry. This category should be used only when 
some positive effort is made to correct; in some instances erroneous 
concepts will be eliminated as tbe students gain an acquaintance with new 
concepts, and in such cases Code 1 would be mor^ appropriate, since the 
elimination of the erroneous concept would really be incidental to the 
primary goal of developing a new concept. 

3., To review concepts previously developed for the primary purpose of 
increasing retention. This v/ould be an objective only after the initial 
deyelopment of the concept (asrepresentedbyCodel) had been completed. 
In some instances, tests may be given for this purpose rather than to 
^aluate concept development, and in cases where a test is given for 
purposes of evaluation as well as to provide overlearning experiences, the 
reporting teacher will have to base his selection of the appr\opriate code 
number on his personal decision regarding die relative priority of the two 
objectives. 



4, To develop a motor or visual-motor -skill to a degree of proficiency that 
makes possible its practical application. This category is appropriate for 
all procedures for initial introduction to the skill to the stage" where 
errorless performance first becomes possible. Correction of performance 
error? is included in this category, 

5, To provide practice in the application of a skill for the express purpose of 
enhancing the probability of its retention. (This category should not be used 
when utilization of the skill is merely incidental to. performance of a 
laboratory exercise which has concept development as its primary goal,) 

6, To develop ah attitude which is clearly a part of the chemistry-course 
objectives. This category would be appropriate when efforts are directed 
toward the development of a personal respect for scientific objectivity; 
it is also appropriate when time is spent in attempts to motivate (i.e„ to 
create a positive attitude in) pupils toward chemistry. 

7, To develop an attitude that i? not directly related to the objectives of the 
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TABLE XXIV (Continued) 



course in chemistry. Time might conceivably be spent in discussing 
attitudes toward minority groups, or in attempting to motivate one or 
more pupils with respect to school work in general, in which case this 
category would be more appropriate. 

8, To evaluate the progress of one or more (or all) pupils in concept 
development. Th^ primary purpose here should be evaluation of progress; 
if increased learning takes precedence, category 1, 2, or 3 should be used, 

9, To evaluate the progress of one or more (or all) pupils in skill development. 
The primary purpose here should be evaluation of progress; if improved 
skill perforihance or enhanced retention of a given level of performance is 
more important than the evaluation process in the teacher’s planning for 
the procedure being reported, then category 4 or category 5 should be used. 

10. The objective of the reported activity was something different from any of 
the above. (E. this category is used, a brief statement of the appropriate 
objective of the activity in question should be made in the remarks section 
or on the reverse side of the reporting form.) 
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(HI); stedent initiated interaction (IV); laboratory work (V); supervised study 
(VI); and miscellaneous and administrative work (VII). When these are compared 



for the two organizational patterns* the expected greater time spent in teacher 



dominated activities in the' large group classes comes^ut clearly (Table XXV). 

» > 

Large group teachers spend more than 43% of the time lecturing* demonstrating, 
showing films, etc. Conventional teachers, however, also spend much time' in 

N. 

' ' ' 

such activities, 33,5% of the time. (Modern ideas of individualizing instruction 



have not reached chemistry classes.) Teacher directed interactions take up the 
next largest time fraction, 33% of the conventional class time, 23% of the large 
group class time. With time spent on evaluation, these two teacher dominated 



activities take up 80% of school time in both kinds of schools. 

■ t? 

On activities in which the student takes the initiative only slight differences 
were noted, L^ge group students spent about 12.5% of their time in student 
initiated in^actions whereas only 10.7% of the time of the conventional 
clas^i^m;^ was available for these activities. Large group students spent 
, about 5% of their time in supervised study agitet about 6% for conventional 
students. Perhaps of some importance is the fact that more than 4% of 
cohventional class time Was devoted to administrative and miscellaneous 

I' 

- ' j 

Chores against less than ^% among the large group schools. It should be noted 



ii 

that our sampling procedure tended to give inadequate weight to the laboratory, 

f 

and that the negligible time reported for it is a product of this factor, 

CMr^d^ta-oixTea^ei’ behaviors confirm certain suspicions but do not give 
conciusive^evidence. They confirm the notion that the personal characteristics 



of the teacher are more significant in determining his style of teaching than 
the organizational pattern in which he finds himself. They confirm that a 
major fraction of the time is spent in teacher dominated behaviors in 
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TABLE XXV 

COMPARISQN OF GROUPED TEACHER BEHAVIORS 

PER CENT OF TIME- 



BEHAVIORS 


LARGE GROUP 


CONVENTIONAL 


Teacher Dominance 


43.5 


33.6 


Evaluation 


13.8 . 


12.2 


Teacher Initiated Interaction 


22.8 


33.0 


Student Initiated Interaction 
( * 


12.5 . 


10.7 


Supervised Study 


5.5 


6.2 


Miscellaneous and Administration 


1,8 


4.3 



o 
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conventional classrooms as well as in large group classes They confirm 

that a sizeable amount of class 'time is spent on administrative and 

miscellaneous routines* The data also indicate that large group students have 

% • 

slightly better access to the teacher for student initiated behaviors, no doubt 
in the small group sessions, and consequently spend somewhat more time 
getting their problems and difficulties discussed. However, aside from these 
blight, inconclusive indications, we did learn a great deal about the teacher 
behaviors under the two organizational patterns* Certainly we did not isolate a 
reason, or even obtain an intimation of a reason, for the consistently greater 
achievement among students in the large group - smalT group organization. 
Unless we accept the extra few per cent of time spent in student initiated 
discussion as the reason for higher achievement^ direct teaching behaviors 
apparently did not influence the results of the study. 

This leads to two hypotheses concerning our findings. Either our data are 
inadequate and we found no differences in the instructional processes because 
our instrument was inadequate to the task, or the differences between large 
group - small group and conventional classes result from the ' pattern of 
organization alone and not from the instructional behaviors under it. We are 
tempted to accept both hypotiieses. We feel that our instrument for recording 
teacher behaviors was not adequate to the needs of the study. Its failure is 

c.- 

unfortunate although not surprising. Researchers have labored for at least* 
50 years on the task of adequately defining teacher behaviors with experiences 
and results similar to ours. In any case, we Judge tEat our instrument was not 
incisive enough to elicit small differeiitiai factors between the two organizational 
patterns which might h^e produced the achievement differences. But we also 
feel that factors related to the organizational pattern alone and independent of 
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teacher characteristics helped produce the results. The change from the tedious 
routine of the academic classroom of 25 pupils, 5 times a week, we suggest, 
produces a “Hawthorne Effect** in the pupils; the demands for individual 
responsibility on the part of the pupil in the large group lecture, we feel, 
leads to somewhat more independent study which carries over into tested 
achievement. Certainly the latter suggestion receives some cor roboration from 
the greater drop-out rate in large group -small group schools in which, we 
think, more dependent pupils find in^equate support to sustain their efforts. 
In summary, our instrument did not show major differences in the instructional 
processes that could be attributed exclusively to organizational pattern; 
however, we believe that the instrument was not adequate to elicit such 
differences should they exist. On the other hand, we propose that the 
instructional behaviors of teachers in the large group -small group schools 
are not the sole cause for the achievement differences we observed and that 
factors such as the novelty and relative impersonality of the large group class 
contribute to produce them. Finally, we want to point to the large differences 
in preparation time which are discussed in the next chapter. No doubt this 
influences the effectiveness of instruction. 
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Chapter 7 
COST 

Educational decisions are never made in terms of a single factor. 
Educational value# post, availability and readiness of staff, community factors, 
and a number of other considerations influence them* Hence, our attempt to 
■ assign specific costs to chemistry instruction, or rather, to determine vdiether 
there were cost differentials between instruction under the large group - small 
group pattern and the conventional pattern constituted a significant part of 
this study* But while cost would appear to be the most readily measurable 
aspect of instruction, it proved quite difficult to pinpoint to an instructional are^ 
School accounting, does not assign specific costs to specific cqurses or 
departments. Consequently, we had to develop a procedure for estimating costs 
related to instruction in Regents Chemistry, 

In actual fact, we were unable to estimate the cost of instruction in 
chemistry in any single school. The best we were able to achieve was a 
reasonable estimate of differentials between schools using the two patterns 
of organization. In order of decreasing amounts involved, such differentials 
depend on differences in the effectiveness of the use of staff, differences in 
the space needs for the two patterns of instruction and the possibility of 
integrating the chemistry space with other instructional space requir^ents, 
differences in the need for capital equipment, and differences in use of 
non-capital instructional supplies. Each of these was considered separately 
in our study. Other cost factors were judged too small and too diffuse and 
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were assumed inconsequential. 

Despite a serious attempt, the^ last item, supplies, could not be isolated 
for comparison. Expendable supplies are typically ordered for the entire 
school or the entire department. Even when items are specifically purchased 
for chemistry courses, they are typically bought in quantities large enough for 
several years, and in a period of expanding enrollments, time averages are 
nearly impossible to comoute. We could not isolate the relevant data. However, 
we could see no ^ priori reason for any cost differentials since both 
organizational patterns were generally similar in their manner of carrying 
out instruction and laboratory work. Also, supplies are a minor fraction' of 
the total budget. Thus, while we were forced to this decision by unavailability 
of data, we feel that we are Justified in assuming that any differentials in 
supplies costs between the two organizational patterns are negligibly small. 

Among the capital items, we found no substantial differences. Most 
schools in the large group organization had purchased overhead projectors, 
but so did many conventional schools. These were used for the chemistry 
instruction and for other instruction in both kinds of schools! The expected 
period of amortization was long so that the annual cost factor became negligibly 
small compared to other cost factors. In capital equipmero^items, also, we 
could not isolate costs specifically for chemistry instruction and we saw nq 
evidence for assuming that appreciable differences existed between the schools 
in the two instructional patterns. 

Space utilization and space cost can be determined quite accurately, 
<• 

Building costs are om record and space use can easily be measured in square 
feet and hours. We attempted an analysis of this kind but found that local 
variables made our data meaningless. Building costs, quite obviously, depend 






om the year of construction* One -school in the study was opened in 1963; 
another had built a science wing two years earlier. Other buildings were built 
in the 50*s, 40^s, 30’s, and 20’s. Some buildings contained additions built at 
various times. 11118 made comparisons useless. But much more difficult to 
analyse are differences quite far removed from the cost of the chemistry 
space. An incisive study of space costs would have required far more time 
and specialized skill than was available for this study. 

The space devoted to science depends on many factors* Some schools 
use separate classroom and laboratory spaces. Some have combined spaces. 
The most recently constructed schools, and one older one, had amphitheater^ 
type classrooms specifically designed for large groups. Two used audltoria; 
one adapted a room for this purpose. We were forced to conclude that taste, 
availability of space, integration into the overall program, and similar non- 
measurable factors determine the space use in a school and that in our limited 
study, it was impossible to assign specific dollar values to either instructional 
pattern. Schools with relatively ample space for science instruction and 
schools with relatively little space existed among both lands* 

Since building costs are a large budgetary factor^ we could not assume 
that- differentials arb negligible although amortization of school plant reduces 
this item also. But we can not express any differentials in more precise terms 
than that space costs seem to reflect the overall philosophy of the school no 
matter what organization is employed for chemistry instruction. 

Only in the area of direct instructional costs and Instructional overhead 

t. V ’ 

costs could we proceed wife our analysis to the point of obtaining theaningful 
data for comparisons. On instructional overhead — supervision and planning — 
we found that only two large group schools expended funds for planning, and 
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these were of tlie order of $300 spent only once, prior to setting up the program. 
No school identified specific administrative or supervisory costs that might 
differentiate large group instruction from conventional instruction. While we 
presume that some additional supervisory work was required in setting up 
large group instruction in all six schools, it obviously was incorporated into 
the normal supervisory and administrative routines. Administrators and 
supervisors, we suspect, simply did not differentiate time spent on setting up 
large group instruction from time budgeted for normal program development.' 
This leaves the largest budget item, instructional salaries, as the sole means 

^ ^ r 

to differentiate costs between the tjvo instructional patterns. 

As we . indicated^ earlier in this report, an adjusted pupil-teacheT load 

rather than salary was used as the basis for teaching cost comparisons because 

/ 

of the extraneous items that determine the exact compensation received by a 

particular chemistry teacher. Salary schedule and fringe benefits differ from 

district to district and in each case reflect the advanced education and 

experience of the teacher. Yet the concern of a comparison of instructional 

costs is with whether more or fewer teachers are required under one pattern 

or the other rather than whether the teacher happens to be high on the salary 

schedule and happens to have advanced degrees or education. 

The adjusted pupil-teacher load was computed from the fraction of the 

% 

class periods actually spent in chemistry instruction, the fraction of assigned/^ 
preparation periods presumably devoted to chemistry (/|)reparations, and 

number of pupils who were instructed in chemistry. Adjustments were n^e 

/ 

for the total number of instructional periods per week in the school and the 
number of periods per week each pupil spends in chemistry instruction. An 
illustration of the computation is given by an example; 
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Pupils: 

Teachers: 



The teacher in this school spends 15 periods teaching Regents Chemistry. 
If we assume that he divides his 12 preparation periods proportionately between 
chemistry and his other teaching assignment of 5 periods, three-fourths of the 
12 preparation periods must also be charged to chemistry. Hence, he devotes 
15 plus 9 or 24 hours to Regents Chemistry. This means that he devotes 24/35 
or .685 of the school week to chemistry. At the same rate of instruction, if he, 
devoted the entire school week to chemistry, he could instruct 78/.685 = 114 
pupils in the 6 period peiL week chemistry course. If the course met only 5 
periods per week, he could instruct 1.2 times as many pupils. Adjusted for this 
factor also, the pupil-teacher load becomes 137 pupils per week, (This lasti 
adjustment was needed to reduce 5, 6, and? period per week chemistry courses 
to a common denominator,) 

Computations of the type illustrated here were made for all large group 
schools (supplemented by one school not in the study which also instr^s" 
under the large group - small group plan) and for all conventiongl-s'ehools in 
the study (supplemented by 12 other schools). Data for the additional schools 
came from returns received to a questionnaire sent to a random selection of 
Western New York high schools. Table XXVI shows the results. The adjusted 
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Number of pupils instructed in Regents Chemistry 78 

1 ^ 

Number of periods per week spent in Regents Chemistry 6 

Number of periods assigned to Regents Chemistry 15 

■J 

Number of periods assigned to other instruction 5 

Number of periods assigned, non-instruction 

tiidy I ©1x3 ♦) 3 

Number of periods assigned to preparation 12 

Total per week 35 



TABLE 50CVI 



ADJUSTED TEACHER-PUPIL LOADS AND 
TEACHER PREPARATION TIME 



■ CONVENTIONAL 


LARGE GROUP 


Number of Schools ^ 


20 


7 


Number of pupils' taking Regents Chemistry 


" tir- 




Range 


18-340 


85-195 


Median 


63 


130 


Range, Adjusted Pupil-Teacher Load 


75-170 


82-204 


Range* Preparation Periods Per Week 


2-11 


5-15 


Range* Percent of Time Available 
for Preparation 


6-28% 


18-43% 


(Preparation Periods/Total 




Periods Per Week) 






■!S 

Mean Adjusted Pupil-Teacher Load 


122* 


130* 


Standard Deviation 


21.1 


39.1 ' 


Meaji Preparation Periods 


6.1** 


10,8** 


Standard Deviation 


2.1 


3.2 


Mean Percent of Time Available 
for Preparation 


16,5%** 


32.3%** 


Standard Deviation 


5.2 


7,5 



*Not significant at 0,5 level 
** Difference significant at 0.0005 level 
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pu|)il-teacher loads do not differ significantly between the large group schools 
and the conventional schools. Neither organizational pattern is more or less 
expensive to staff than the other because teachers under both instruct Just 
about the Same number of pupils. However, in preparation time, the large 
group teachers fare significantly better. Nearly one-third of their working 
time is available for preparations (vhich may be one important factor in the 
higher achievement of their pupils) in contrast to one-sixth of the time for 
conventional teachers. While the range of preparation time overlaps in the 
table because one of the seven large^group teachers had much less preparation 
time than his colleagues, all remaining large group teachers had more 
preparation periods than any conventional teacher. (It should be emphasized 
that the tests of significance are merely indicative, not confirmatory, because 
of sampling problems,) 

CXir analysis of costs under the two instructional patterns leads to the 
following conclusions. There is no difference in the cost of instruction under 
the large group - small group or the conventional pattern. Cost of space seems 
to be determined by factors other than organizational pattern. Teachers under 
the large group - small group organization have substantially more time for 
preparation without diminishing their service in terms of numbers of pupils. 
Cost is not a factor which should influence toward or away from large group - 



small group instruction. 



Chapter 8 

RECOMM'ENDATIONS 



, The evidence gathered in this study is relatively self-explanatory. On 
13 achievement measures corrected for initial pupil differences, pupils taught 
under the large group -small group instructional patterns surpassed pupils 
taught conventionally to statistically significant extents. This finding was 
applicable to students who initially scored high on the control measures as 
well as to pupils who scored low on these measures. It applied to boys as well 
as girls (with some exceptions). On measures of interest, no significant 
differences between large group - small group taught pupils and conventionally 
taught pupils were found. Also, no cost differences were found. between the two 
organizational patterns while large group teachers had additional preparation 
time. Hence, it appears that large group -small group instruction is an 
educationally desirable pattern of instruction. 

Nevertheless, we can not, on the basis of this study, recommend that all 
schools adopt large group -small group instruction without pointing very 
specifically to a number of limitations of this study which reduce the general 
applicability of its findings. 

Sampling Problems 

The first limitation of our results is the nature of the sample. Schools 
which participated in this study did so on their own initiative. They agreed to 
support the study financially which implies a research orientation among their 
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teachers, admiiiistratlve staffs, and boards. They were primarily suburban 
schools. The assumption of random sampling of schools in the study is 
definitely not tenable. 

Secondly, the large group - small group schools had decided on that pattern 
of instruction independent of the study. Their teachers, administrators, and 
boards were therefore distinguished from those in the conventional schools by 
their prior independent decision to try large group instruction. 

Thirdly, alL teachers in the schools which participated had been identified 
by the -State Education Tejiartment as having substantial experience with 

Regents Chemistry and having in the past had classes who did well on the 

' • \ 

Regents Examination, Hence, the findings are definitely limited to experienced 

teachers who have successMly taught the Regents %llabus. 

Because of the restrictions of the sample, any application of this study is, 

in our view, limited to schools in which parallel conditions hold. We feel diat 

/ 

the achievement results obtainedhere have aMgh probability of being duplicated 
only in suburban schools with experienced and successful teachers after-the 
staff, administration, and boards have reached the decision, on other bases, 
to try large group - small group instruction. 

Presumed Success factors 

'■ second set of cautions relates to our inability to define precisely 
differences in behavior between teach ^rs in the two organizational patterns. 
This forces us to assume that the organizational pattern, in and of itself, was 
the determining factor in the observed differences. Consequently we must 
hypothesize about the importance of the large group -small group itself. 
All the large group schools used that pattern of organization in courses 



o 
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otlier than chemistry, hut in no school was the large group -small group used 
extensively. Most schools used it only for one grade level of history instruction. 
Hence, the findings, it seems to us, are dependent on the fact that 'the large 
group -small group pupils found - the pattern somewhat of a novelty. We 
hypothesize that this novelty was important (the Hawthorne Effect), We would 
be seriously concerned if our findings were generalized to other subjects with 
the result that students experienced few or no conventionally taught classes in 
the later high school years. Entirely different results might then occur. 
Secondly, we feel that the extra preparation time of teachers In the large 
group small group pattern has a causal relation to the higher achievement* We 

strongly suspect that erosion of the extra preparation time by administrative 

/' 

decision, after the newness of tiiis organizational pattern fades and it loses its 
experimental designation, would produce different results. We found educational 
value to large group - small group instruction, not lower costs* We fear that an 

o 

attempt to add saving as a second value,, by reducing the preparation time, 
would lead I to the sacrifice of its ‘educational values. 

Pinally, we assume that a reason for the higher achievement in large 
group -small group instruction was the fact that students could not depend as 
much on their teachers for instruction but had to rely on their own studies to 

\ « r 

a"-greater extent. This is, of course, a double value: greater achievement and 
greater independence ‘on the part ofthe student. Yet, this factor had its negative 
aspects as well. Some students withdrew from chemistry, presumably, because 
they were unable to make the adjustment. These students constitute a loss of 

• V I . 

undetermined magnitude. Regents Chemistry is normally taught in^lXth grade. 
Pupils in this age group can be expected to show considerable maturity, yet 
they are still late adolescents. We do not know v^ether the withdrawals were of 
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students who learned their limitations early rather than later, in college 
perhaps, or of shidents who needed additional time to mature into -independence. 
We suspect both groups were represented and are concerned about the loss of 
the latter. 

Recommendations ' ' ' 

In view of these cautions, our recommendations are conditional. If a 
school staff, admi^tration, and school board reaches the decision to instruct 
by the large group Hmall group organizational pattern, and it is in a suburban 
area and has an experienced and successful chemistry teacher, there is a high 
probability that achievement in the Regents Chemistry course will increase 
and that the instruction will cost no more than under the conventional pattern* 
However, the achievement gains will probably be accompanied by a greater 
withdrawal rate from chemistry. The gains will be limited to academic 
achievement and will not include m increase in interest or a differential gain 
in science reasoning and_ understanding. Moreover, we feel that these gains 
are predicated on the fact that the large group teacher has substantially more 
time available for preparation. If these conditions are met, we definitely feel 
that we can recommend the decision to adopt large group -small group 
instruction as an educationally sound decision. 



Chapter 9 

A NOTE ON PROCEDURE 



The research descr ibed in this report is not an experiment in the classical 
sense of the term. We. did not randomly choose and assign treatments and in 
several other respects we violated the norms of experimental design. It was 
an applied, not a theoretical investigation and we are well aware of its limited 
character. Our study clearly reflects the complex factors that are involved in 
the improvement of education through research (Brickell, 1961). Clearly we 
operated in the realm of application rather than in the realm of fundamental 
investigation. The character of the problem, however, demanded such an 
approach. 

Large group -small group instruction is a phenomenon of considerable 

' t ' 

current interest. It is^eing applied in schools for a number pf reasons although, 
as we indicated earlier , without adequate knowledge of its true functioning. The^ 
first questions to be asked about it, it seems to us, is not why does it work or 
even how does it work, which would be questions tiiat would require detailed 
experimental investigation, but rather does it work? And it was to the latter 
question that we. devoted all efforts. 

From the purely practical point of view^ from the point of view of either 
encouraging or discouraging school administrators in the use of this pattern, 
the latter question seems to us by far the most important. It would be redundant 
to spend three years on a detailed investigation of how large group instruction 

• * i 

works only to find out, at the end, that it is detrimental to the student. Obviously 



a school administrator wants to know first that it works before he would try it 
in his school. 

It seems to us, likewise, that our research would have limited generality 
and applicability in schools if our procedures had demanded a set pattern of 
behaviors among the large group teachers. From a theoretical point of view, 
it is cf course extremely desirable to know precisely what kinds of behaviors 
are likely to lead to optimum results. Then, teacher training efforts can be 
directed to developing these behaviors. (Although this goal has been pursued 
in research for at least 50 years, it has produced extremely meager results. 
We still do not know the behavior of . a good teacher and indeed we do' not even 
know how to define him.) But to demand that large group teachers who wish to 
duplicate our results change their teaching practices to a specific set of 
behavior patterns, or even to train experimental teachers in these patterns 
when we do not know which behaviors are likely to be effective, seemed to us 
an impractical procedure. In view of these circumstances, we felt that a 
restriction on the performance patterns of our large group - sihail group ' 
teachers would limit the usefiilness of our findings rather toan increase our 
understandings. 

The application of dur results involves at least three sets of probabilities 
all of which must be fairly high or our study is extremely restricted in its 
usefolness* The first set of probabilities relates to the statistical significance 
of our results. Here we found that the probabilities are extremely large that 
schools similar to those in our study would find increased achievement through 
large group instruction. The next set of probabilities relates to finding schools 
similar to those in our study. In this regard our sarnie was restricted to 
suburban schools with experienced and successful chemistry teachers. The 
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probability that our findings will be replicatec^ is therefore restricted to 
similar schools. This fits a substantial fraction of the schools in New York 
State or in the nation and consequently the probability of finding similar 
schools is fairly high. The next probabilities relate to the instructional 
behaviors of the experienced and successful teacher in those comparable 
schools. Because of the design of our study, this probability is extremely hi^ 
also because we placed very few, if any, restrictions on the teacher behaviors, 
Both^^ the specific school arrangements for large group instruction — time and 

f 

spaces provided-— and the teacher behaviors were left entirely to the discretion 

of the particular school and the particular teacher. Consequently we cap. feel 

that most arrangements for teaching large groups of pupils, with associated 

small group and laboratory meetings, and most indivhiual styles for teaching 

which do not deviate extremely far from the more or less customary patterns 

represented in our study would meet this criterion at an extremely high level* 

Thus we feel that our design maximized all three sets of probabilities governing 

> 

the reproduceability of our results and, as a consequence, produces findings of 
fairly high generality and usefulness. 

-We designed our study in order to provide maximum information of a 
practical sort to school administrators desiring to try large group- small 
group instruction in their schools. This design forced us into, ap empirical 
procedure that violated some of the canons ol^basic research. It precluded our 
testing of certain hypotheses regarding fundamental relationships under large 
group -small group pattern and, for that matter, of contributing much to basic 
knowledge regarding the learning process. On the other hand, it gives some 
relatively cleah'cut answers to the practical question; “What is the likelihood 
-'■^’hat some specific suburban school would benefit from the adoption of large 




r . 

■ ' group -small group instruction in chemistry?” The administration of that 

school must judge to what extent ^achers and school are like the teachers and 
. schools of our study* If the similarities are large, they can have considerahle 

faith that pupils in their school, too, will attain higher achievement under large . 

0 ^ 

group ~ small group instruction. 
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